Immersion cooling for AI GPUs: how it works and why it matters

Why AI GPUs broke air cooling

For decades, data center cooling was an air-handling problem: cool air in at the front of each server, warm air out at the back, with aisles, air handlers and chillers doing the rest. It worked because rack loads stayed within what moving air could absorb.

AI changed the equation. Each accelerator generation packs more power into a similar package, so the heat flux at the chip keeps rising. Clusters then place GPUs as close together as possible, because shorter interconnects mean faster communication. The result is a great deal of heat in a small footprint, precisely what air handles worst.

Air holds little thermal energy per unit of volume, so removing more heat means moving far more of it: larger heatsinks, faster and louder fans, colder supply air. All of that consumes energy without adding any compute. Past a certain density, the limit of a rack is no longer the silicon but the airflow needed to keep it within temperature. Liquid cooling brings the coolant to the heat instead. Direct-to-chip cold plates are one route; immersion goes further and places the entire server in the liquid.

How single-phase immersion cooling works

In a single-phase immersion system, servers stand vertically in a tank filled with a dielectric fluid, a liquid that does not conduct electricity. GPUs, CPUs, memory and power components run fully submerged, in direct contact with the coolant. “Single-phase” means the fluid stays liquid throughout: it warms, circulates and cools, but never boils. The heat path runs in five steps:

1. Heat enters the liquid. Every component releases its heat directly into the surrounding fluid.
2. Warm fluid is collected. A return manifold sends the warmed liquid to a coolant distribution unit (CDU).
3. The CDU exchanges the heat. A heat exchanger transfers it to a separate facility loop. The two fluids never mix.
4. Cooled fluid returns. A distribution manifold feeds the cooled liquid back evenly across the servers.
5. Heat is rejected outside. The facility loop carries the heat to heat-rejection units, such as roof-mounted coolers, which release it into the ambient air.

Pumps move the liquid; the servers have no fans. In effect, the tank replaces the hot and cold aisles and most of the air handling of a conventional data hall.

Single-phase vs. two-phase immersion

Two-phase immersion uses a fluid with a low boiling point. It boils on hot components; the vapor rises, condenses on a cooled coil above the bath and drips back. Phase change absorbs a great deal of energy, which makes two-phase very effective at hot spots. In return, tanks must be sealed to contain the vapor, fluid losses need tight control, and some fluorinated fluids historically used for two-phase cooling face growing regulatory scrutiny.

Single-phase immersion keeps the fluid liquid at all times and relies on circulation and a heat exchanger. The fluids are typically engineered hydrocarbon-based or synthetic liquids with a high boiling point, so evaporation is minimal and there is no vapor to contain. Servicing a server means lifting it out of a liquid bath, not opening a sealed vapor environment.

Both can cool dense hardware. For a system shipped to remote sites and serviced by field engineers, single-phase is the simpler, more forgiving choice, and the one used in the BAYWAW AI Container.

What immersion cooling changes

Energy efficiency

Industry-average PUE has hovered around 1.5–1.6 for years, according to the Uptime Institute’s annual Global Data Center Survey. Put simply, for every kilowatt-hour that reaches the IT equipment, a typical facility spends roughly another half kilowatt-hour on cooling and other overhead. The BAYWAW AI Container has a PUE of 1.05. Immersion closes most of that gap because pumping a liquid through a heat exchanger takes far less energy than chilling and moving large volumes of air. Removing server fans also lowers the IT load itself, a saving PUE does not even capture. At the scale of a 1.2 MW facility running around the clock, roughly 10.5 GWh a year, the gap between an industry-average PUE and 1.05 is a line in the operating budget, not a rounding error.

Density

With heat removed at the source, servers sit side by side without the spacing and ducting that airflow demands. In the BAYWAW AI Container, that means 104 NVIDIA B300 GPUs per tank and 624 in a single 40ft ISO container that travels by road or sea.

Reliability

There are no server fans, moving parts that wear and fail over time. Components sit beneath the liquid surface, shielded from dust and humidity, which also limits corrosion. The thermal mass of the bath smooths temperature swings, so hardware runs in a steadier environment than air usually provides.

Climate

Desert heat, sandstorms and coastal humidity are hard on air-cooled hardware. In a tank, the electronics never touch the outside air; only the heat-rejection units face the environment, and they can be engineered for local conditions. That matters across the Gulf and MENA, where many sites chosen for sovereign AI infrastructure combine high ambient temperatures with airborne sand.

Noise

AI servers under load in an air-cooled hall are loud. Without server fans, an immersion tank is quiet; the main moving parts are the pumps and the roof-mounted coolers.

Operations and maintenance: the honest trade-offs

Immersion changes how hardware is serviced, not the rigor it requires.

• Lifting blades. Servers are lifted vertically out of the bath and left to drain before any work begins, a different gesture from sliding a server out of a rack.
• Fluid management. Levels are monitored, small losses during service are topped up and the condition of the fluid is checked over time.
• Training. Technicians need specific training in immersion hardware and fluid handling, and the pool of experienced engineers is smaller than for air cooling.
• Warranty coordination. Server warranties were historically written with air cooling in mind; for immersion, terms have to be agreed with the original equipment manufacturers.

None of this is a reason to avoid immersion, but it is a reason to choose an operating model before choosing a tank. At BAYWAW, these tasks sit inside our managed services: on-site engineers handle blade servicing, component replacement and the fluid; warranty management covers OEM coordination and claims; and 24/7 monitoring watches the thermal chain. Your team does not have to handle the liquid.

How BAYWAW implements immersion cooling

The BAYWAW AI Container applies single-phase immersion inside a 40ft High Cube container, built, integrated and tested in the UAE.

• Six GRC NX immersion tanks, arranged 3 + 3 on either side of a central service aisle, so every tank is within reach.
• 13 servers and 104 NVIDIA B300 GPUs per tank, for 78 Hypertec servers and 624 GPUs in total.
• A clear tank anatomy: a coolant distribution manifold, server blades on liquid-cooled plates submerged in the dielectric bath, and a return manifold back to the CDU.
• Redundant coolant loops and CDUs, so cooling does not hinge on a single unit, with roof-mounted coolers rejecting the heat.

Tanks, CDUs and coolers are pre-integrated and tested together, so cooling arrives as part of the container rather than as a plant to build on site. The container can be operational in as little as four months from order, on grid power or a solar farm with battery storage. For the full heat path, see our immersion cooling technology page, or go inside the BAYWAW AI Container.

Air cooling vs. immersion cooling at a glance

Air cooling remains a sound choice for general-purpose IT; the balance shifts as density rises. This qualitative comparison assumes dense AI GPU servers.

Conclusion: cooling is now a compute decision

With AI GPUs, cooling decides how much compute fits on a site, how much of the energy bill reaches the silicon and how steadily that silicon runs. Single-phase immersion addresses all three, at the cost of new maintenance habits best left to a dedicated team. For the wider decision, read our containerized data center buyer’s guide.