Cooling: Air, Water and Liquid to the Chip

Almost every watt that goes into a server comes out as heat. A 100 MW data hall is, from a physics point of view, a 100 MW space heater that happens to compute along the way. Cooling is the job of moving that heat from the chip, out of the building, and into the outdoor air or a body of water. Every method makes a trade between electricity and water, and the right choice depends on climate, rack density and what the local water supply can bear.

How hot is too hot?

ASHRAE, the engineering society that publishes the standard thermal guidelines for IT equipment, recommends server inlet air between 18 and 27 degrees C (about 64 to 81 degrees F). It also defines wider "allowable" ranges; Class A1 equipment can run from 15 to 32 degrees C. Older data centers were kept chilly, around 20 degrees C or lower, out of habit. Running warmer saves cooling energy, and most modern sites run near the top of the recommended range.

Air cooling and aisles

Servers pull air in the front and blow it out the back. If racks all faced the same way, each row would breathe the previous row's exhaust. So rows face each other: fronts together make a cold aisle, backs together make a hot aisle. Adding doors and a roof to one aisle, called containment, keeps the streams from mixing and lets the cooling system work with warmer supply air.

Air has a limit. It carries little heat per unit volume, so cooling a 100 kW rack with air alone would take a hurricane of fans. In practice, air cooling tops out somewhere around 30 to 50 kW per rack, even with containment and rear-door heat exchangers. Past that, liquid has to come closer to the chip.

Getting the heat outdoors

Inside the hall, heat moves into air or liquid. Outside, it has to leave the site. The main options:

Heat rejection method	How it works	Water use on site	Electricity use	Best climate
Direct or indirect evaporative cooling	Outside air is cooled by evaporating water, directly or across a heat exchanger	High in hot, dry weather; near zero in cool weather	Low	Dry climates
Water-cooled chillers with cooling towers	Chillers cool a water loop; towers evaporate water to dump the chillers' heat	High, year round	Moderate	Most climates, where water is available
Air-cooled chillers	Chillers reject heat through fans and coils, no evaporation	Very low	Higher, especially on hot days	Water-scarce areas
Dry coolers (free cooling)	Fans blow outdoor air across coils carrying warm coolant; no compressor when outside air is cool enough	Very low	Low when cool outside, needs backup on hot days	Cool climates, or warm-water liquid cooling anywhere
Outside air economizer	Filtered outside air goes straight into the hall when conditions allow	None (or a little for humidity)	Very low	Cool, clean-air climates

Liquid to the chip

Direct-to-chip cooling bolts a metal cold plate onto each processor. Coolant flows through it, picks up heat right where it is made, and returns to a coolant distribution unit, which hands the heat to the building's water loop. Nvidia's newest rack designs run almost entirely on this approach, with coolant entering at up to about 45 degrees C. Warm coolant is the quiet breakthrough: water that leaves the racks well above outdoor temperatures can often be cooled by plain outdoor air, without chillers and without evaporating water.

Immersion cooling drops whole servers into tanks of non-conductive fluid. Single-phase immersion circulates the fluid through a heat exchanger. Two-phase immersion uses a fluid that boils at a low temperature and condenses on coils. Immersion is common in crypto mining (Riot Platforms' Rockdale, Texas site uses both immersion and air cooling) and is growing in some AI and HPC deployments, but it requires special servers and handling, and some two-phase fluids face scrutiny over PFAS chemistry.

Water: direct and indirect

A data center uses water two ways. Direct water is what the site itself consumes, mostly through evaporation in cooling. Indirect water is consumed at power plants generating the site's electricity, mostly through steam cycle cooling at thermal plants. Lawrence Berkeley National Laboratory estimated that U.S. data centers directly consumed about 17 billion gallons in 2023, and that indirect water tied to their electricity was roughly 12 times larger. Its scenarios had direct use possibly doubling, or even quadrupling, by 2028. Indirect water depends on the local grid mix: wind and solar use almost none, thermal plants use a lot.

Companies are moving away from evaporative cooling in water-stressed places. Microsoft reported a fleet average water usage effectiveness of 0.30 liters per kWh in fiscal 2024, down from 0.49 in 2021, and said all new designs since August 2024 use closed-loop cooling that is filled once and recirculated, avoiding more than 125 million liters a year per datacenter. Its Fairwater AI campus in Wisconsin uses closed-loop liquid cooling. The trade-off, which Microsoft acknowledged, is somewhat higher electricity use, which it described as nominal across the fleet.

Cooling approach	Rack density it handles	Site water	Cooling power
Room air, no containment	Up to about 5 to 10 kW	Depends on heat rejection	Highest per kW of IT
Air with hot or cold aisle containment	Up to about 20 to 30 kW	Depends on heat rejection	Moderate
Rear-door heat exchangers	About 30 to 50 kW	Depends on heat rejection	Moderate
Direct-to-chip liquid	100 kW and up	Can be near zero with warm water and dry coolers	Low (fans shrink, chillers often avoided)
Immersion	100 kW and up	Can be near zero	Low