NVIDIA data center hardware is cooled with water ‘hotter than a spa’

Here’s a question you probably didn’t expect to read today (or ever): What do AI supercomputers and spas have in common? Sounds like the start of a joke, doesn’t it? The truth, however, is that even if a spa operates between 38 and 40 degrees Celsius, NVIDIA’s latest AI servers push their “coolant” liquid even hotter, up to 45 degrees Celsius, or 113 degrees Fahrenheit.

This may seem backwards, because how can a hot liquid cool server rooms? The old approach was for data centers to pour tons of power into cooling air, then push the cold air into every component, running noisy fans and coolers just to keep the chips comfortable. Now, when the coolant gets hot, outdoor dry coolers can remove the heat themselves without the need for coolers. Once you understand how it all works, you’ll find that the old method of data center cooling is pretty useless.

NVIDIA’s Rubin-generation AI infrastructure is entirely liquid-cooled. No fans, no cold aisle, no going through a freezer when you’re in the server room. Just liquid flowing quietly in a closed loop. This is a major update because, historically, cooling has eaten up as much as 40% of a data center’s electricity bill. Industry estimates indicate that increasing chiller temperature by just one degree reduces cooling energy costs by approximately 4%. By scaling that up to a 50-megawatt installation, you get about $4 million in energy savings per year.

The water savings are just as striking. Conventional cooling tower systems can consume approximately 2.6 million gallons of water per megawatt per year. NVIDIA’s new 45-degree liquid cooling architecture can bring that number down to almost zero.

This is a complete overhaul of the cooling system of large data centers. A closed-loop system without evaporative cooling, water towers, and continuous water drainage will not only impact data center costs, but also the environment.

For decades, the data center industry has assumed that cold means security. The cooler the room, the better. But silicone chips are naturally hot. NVIDIA data shows that coolant entering a fully liquid-cooled processor at 45 degrees Celsius comes out at around 55 degrees after absorbing the thermal load of the chip. The chip continues to operate at full power because the liquid-cooled cold plates keep everything within safe operating limits. This also allows outdoor dry coolers to effectively remove heat for most of the year, without the need for mechanical coolers. Of course, geography also plays an important role here, because one cannot have the same expectations regarding temperature management in the desert as in the mountains.

This will also result in quieter data centers. Normally, you need hearing protection inside data centers to reduce loud noises produced by cooling fans, backup generators and cooling systems. NVIDIA’s Rubin architecture abandons all that.

NVIDIA’s Rubin equipment doesn’t just reduce energy and water consumption costs and reduce noise levels. Servers are also more space efficient. Their design means a system that previously took up six rack units can now hold two.

Ultimately, NVIDIA’s new server architecture has the potential to solve some of data centers’ most annoying problems, such as their high power consumption, massive water consumption, noise pollution, and land use. This does not solve the problems of existing data centers, but we hope that it will lead to better centers in the future.