Computing Power Liquid Cooling and AI Servers

Liquid cooling is essential for modern AI servers, enabling high-density GPU performance, energy efficiency, and sustainable data center operations.

Why Liquid Cooling is Critical for AI Servers

AI servers, particularly those used for training large language models and other compute-intensive workloads, generate extreme heat due to high Thermal Design Power (TDP) GPUs and CPUs. Traditional air cooling becomes insufficient when rack power densities exceed 20–40 kW, with some AI racks reaching 100–227 kW per rack . Liquid cooling efficiently removes heat directly from processors, stabilizing temperatures, preventing throttling, and allowing higher computational density .

Types of Liquid Cooling

1. Direct-to-Chip (D2C) Cooling
   • Uses cold plates attached to CPUs or GPUs.
   • Coolant flows through channels in the plate, absorbing heat at the source.
   • Variants include single-phase (coolant remains liquid) and two-phase (coolant vaporizes to absorb heat) systems .
   • Captures up to 80% of server heat, with remaining heat often managed by air cooling.
2. Immersion Cooling
   • Servers are fully submerged in dielectric fluid.
   • Provides uniform cooling across all components.
   • Can capture 100% of heat and enables high-density racks with potential for heat reuse .

Cooling Infrastructure Components

• Coolant Distribution Units (CDUs): Regulate temperature, flow, and pressure of the coolant, isolating IT loops from facility water systems .
• Heat Rejection Systems: Can be liquid-to-air, liquid-to-liquid, or refrigerant-based, depending on facility design and density requirements .
• Hybrid Environments: Many data centers combine air and liquid cooling to retrofit existing facilities or optimize energy efficiency .

Benefits of Liquid Cooling

• Performance: Maintains optimal GPU and CPU temperatures, enabling sustained high-speed AI workloads .
• Energy Efficiency: Reduces Power Usage Effectiveness (PUE), lowers fan energy consumption, and allows higher operating temperatures for air systems .
• Sustainability: Enables heat reuse for building heating or industrial processes, reduces water consumption, and supports renewable energy integration .
• Space Optimization: Fewer racks are needed for the same compute power, reducing physical footprint .

Real-World Implementations

• ASUS AI POD with NVIDIA NVL72 GPUs: Fully liquid-cooled rack-scale systems handling up to 227 kW TDP, enabling trillion-parameter AI models .
• Applied Digital Ellendale Campus: Uses direct-to-chip cooling at scale, achieving high-density AI infrastructure in a compact, energy-efficient design .
• HPE Cray Liquid-Cooled Supercomputers: Demonstrated 87% reduction in CO2 emissions and 86% lower annual cooling costs compared to air-cooled equivalents .

Integration and Planning

Deploying liquid cooling requires careful planning:

• End-to-End Design: From grid to chip and chip to chiller, integrating power and cooling systems ensures reliability and efficiency .
• Global Manufacturing and Lifecycle Support: Scalable deployment depends on coordinated supply chains and digital modeling tools for simulation and predictive maintenance .
• Flexibility: Data centers often mix air and liquid-cooled servers, allowing gradual upgrades and hybrid operation .

Conclusion

Liquid cooling has transitioned from a niche HPC solution to a critical enabler for AI data centers, supporting extreme GPU densities, improving energy efficiency, and enabling sustainable operations. Direct-to-chip and immersion cooling, combined with integrated CDUs and heat rejection systems, allow modern AI servers to operate at peak performance while reducing environmental impact and operational costs .