Nvidia builds Selene, the 7th fastest supercomputer in the world

Nvidia had completed building the 7th faster supercomputer in the world in just one month. Named Selene, the machine is quite unique owing to the fact that it houses GPU-accelerated DGX SuperPOD architecture from Nvidia, which is available commercially, rather than the customer CPU-heavy design seen across most of the Top 500 list.

Further, the peak theoretical performance of Selene is pegged at just under 35,000 teraflops. It has also been ranked second in the Green500 list of the most power-efficient supercomputers. It boasts of 560 AMD Epyc 7742 CPUs having 64 cores each, as well as 2240 Nvidia A100 GPUs.

Apparently, the previous supercomputers built by Nvidia took months to complete, besides being extremely difficult to upgrade or maintain. As such, while preparing the design for Selene the company aimed for making it as simple and modular as possible. The machine’s 280 nodes are standardized DGX pod each, containing eight Nvidia A100 GPUs as well as two AMD Epyc CPUs.

It is cooled on a per-SuperPOD basis. Every SuperPOD resides in one big air conditioned warehouse, raised off the ground and having fans underneath for pushing cool air up in the pods. The small assembly team of Nvidia only required to install the flooring and seal up the SuperPODs for controlling the air flow.

The homogeneity of Selene allowed to be assembled this fast. It primarily involved moving every DGX into the correct position and wiring them, checking that they worked. Wiring a supercomputer is usually hard, but Nvidia leveraged Mellanox’s InfiniBand switches for reducing the number of cables needed, while increasing the bandwidth simultaneously.

Nvidia was creative in monitoring equipment for Selene. They bought Trip, a little robot, that can be controlled remotely and moved around to observe the goings-on inside Selene. They also developed a bot for Slack, sending them notifications when the hardware is misbehaving or when a cable has come loose.

 

Source Credits: https://www.techspot.com/news/86390-nvidia-assembled-world-7th-fastest-supercomputer-one-month.html