Inside the Building: White Space, Racks, Servers and GPUs

Walk into the main hall of a data center and the first thing you notice is the noise: thousands of fans pushing air, a steady roar that makes conversation hard. The second thing is the order. Rows of identical black cabinets, cables bundled in trays overhead, painted lines on the floor. This room is the white space, where IT equipment lives. Everything that supports it, the switchgear, batteries, chillers and generators, sits in gray space: back rooms, mechanical galleries, rooftops and yards. In a big modern campus the gray space can take as much land as the white space.

The rack

A rack is a standard steel frame 19 inches wide inside, usually 42 to 48 rack units tall. One rack unit, written 1U, is 1.75 inches. A typical general-purpose server is 1U or 2U, so a rack might hold 20 to 40 servers plus a network switch at the top and two power strips (called PDUs, power distribution units) running down the back. Two power strips fed from separate electrical paths let a server keep running if one path fails.

The number that matters most is rack density: how many kilowatts a rack draws. Uptime Institute's 2025 survey put the average typical rack at almost 9 kW, up from 8.3 kW in 2024. About three quarters of operators said their typical rack runs under 10 kW. Only a few percent typically run 30 kW or more, though the share with at least some racks above 30 kW is growing.

What goes in the racks

• General-purpose servers. Each has one or two CPUs, memory and local drives. They run websites, databases, email and business software. A typical one draws a few hundred watts to about a kilowatt.
• GPU servers. Packed with accelerator chips. Nvidia's H100 SXM chip is rated up to 700 W on its own. A common eight-GPU server draws around 10 kW, about what a whole rack of ordinary servers used to.
• Storage. Arrays of hard drives and flash drives that hold the data. Hard drives still store most of the world's bulk data because they are cheap per terabyte; flash handles the fast work.
• Network gear. Switches at the top of each rack connect to bigger switches at the end of each row and in a central core. AI clusters add a second, much faster network just for GPU-to-GPU traffic.

Why AI racks are so hot

AI training splits one huge job across thousands of GPUs that trade results constantly. The shorter the cables between them, the faster they can talk, so chip makers have pushed to put as many GPUs as possible in one rack. Nvidia's GB200 NVL72 joins 72 Blackwell GPUs and 36 Grace CPUs in one liquid-cooled rack. Industry estimates put racks in this family at well over 100 kW. Integrator 2CRSi lists the newer GB300 NVL72 at about 135 to 140 kW and Vera Rubin NVL72 racks at roughly 190 to 230 kW, with 600 kW racks on Nvidia's roadmap for 2027. Vendor figures depend on configuration, so quote them as approximate.

Era or rack type	Typical power per rack	Cooling
Enterprise rack, around 2010	About 3 to 5 kW	Room air
Average rack, Uptime 2025 survey	Almost 9 kW	Mostly air
Dense cloud or early GPU rack	15 to 40 kW	Air with containment, sometimes rear-door coolers
Nvidia GB300 NVL72 class	About 135 to 140 kW	About 90 percent direct liquid
Nvidia Vera Rubin NVL72 class	About 190 to 230 kW	Fully direct liquid
Announced roadmap	600 kW and up	Liquid, with new power designs

The network inside the hall

Ordinary cloud halls use a layered network: a switch at the top of each rack, links up to bigger switches serving a row or a pod, and a core that connects pods to each other and to the outside world. Copper cables handle the short runs inside a rack. Fiber takes over for anything longer than a few meters. A large building can hold tens of thousands of fiber strands in overhead trays.

AI halls add a second network just for GPUs. Inside an NVL72 rack, the GPUs talk over Nvidia's NVLink connections, which act like one giant switchboard. Between racks, clusters use very fast Ethernet or InfiniBand links, often 400 or 800 gigabits per second per port, wired so that any GPU can reach any other in a few hops. That second network can cost as much as a meaningful share of the cluster and draws its own power. It is also why AI training wants every GPU in one building or one campus: stretch those links across a state and the chips spend their time waiting.

Storage sits on its own network too. Training jobs read huge datasets and save checkpoints, snapshots of the model's progress, every so often, so that a failed chip does not mean starting over. A single checkpoint for a very large model can run to terabytes.

Weight is a problem too. A loaded AI rack can weigh well over a ton, more than many older raised floors were built to carry. New AI halls often put racks directly on a reinforced slab and run power and cooling pipes overhead.