What types of liquid-cooled servers are available for AI

Liquid cooling is essential for modern AI servers, enabling high-density workloads, improved thermal efficiency, and energy savings beyond the limits of traditional air cooling. Why Liquid Cooling is Critical for AI ServersAI servers, especially those used for training large language models or high-performance computing, generate extreme heat, often exceeding 80 to 120 kilowatts per rack and sometimes reaching 140 kW in advanced setups. Traditional air cooling cannot efficiently manage these heat densities, making direct-to-chip liquid cooling or immersion cooling necessary.

Liquid cooling stabilizes GPU and CPU temperatures, allowing higher performance and preventing thermal throttling. Key Components and ArchitecturesLiquid-cooled AI servers rely on specialized infrastructure: Cold Plates: Mounted directly on CPUs and GPUs to draw heat efficiently.

In-

Rack Manifolds: Distribute coolant to each server, ensuring leak-proof connections. Coolant Distribution Units (CDUs): Regulate temperature, flow, and pressure of the coolant, isolating the IT fluid loop from the rest of the cooling system. Rear Door Heat Exchangers (RDHx): Used in hybrid systems to remove heat from high-density racks.

There are two main liquid cooling approaches: Direct-to-Chip Cooling: Coolant flows directly over processors and accelerators, ideal for racks above 100 kW. Immersion Cooling: Servers are submerged in dielectric fluids, suitable for extreme densities above 175 kW.

Benefits of Liquid Cooling

Higher Thermal Efficiency: Removes heat more effectively than air, allowing denser server configurations. Energy Savings: Reduces fan usage and overall power consumption. Improved Reliability: Maintains stable temperatures for GPUs and CPUs, enhancing performance and lifespan.

Reduced Noise and Water Usage: Some architectures lower sound levels and water consumption compared to traditional HVAC systems.

Challenges and Considerations

Secondary Component Cooling: Liquid cooling is precise, focusing on CPUs/GPUs, which can leave memory modules, SSDs, and other components in stagnant thermal zones. Infrastructure Requirements: Facilities must accommodate CDUs, piping, and heat rejection systems, which can increase capital costs. Environmental Impact: Liquid cooling increases water consumption, as heat is dissipated through cooling towers, raising sustainability concerns.

Market Trends

The adoption of liquid cooling is accelerating due to AI workloads. As of early 2024, 22% of data center operators were using direct liquid cooling, and the market is projected to grow rapidly, with forecasts of 30%+ annual growth in liquid cooling adoption. Despite this, less than 3% of servers worldwide are currently liquid-cooled, highlighting significant growth potential.

ConclusionLiquid-cooled servers are becoming indispensable for AI data centers, enabling high-density, high-performance workloads that air cooling cannot support. While they offer substantial benefits in efficiency, reliability, and performance, careful planning is required to address infrastructure, secondary component cooling, and environmental impacts. Collaboration among data center operators, system integrators, and cooling vendors is essential for successful deployment.