AWS unveils custom GPU cooling system for Nvidia AI servers

Amazon Web Services (AWS) has developed a hybrid liquid-air cooling system called the in-row heat exchanger (IRHX) to manage thermal loads from Nvidia’s latest GPUs used in AI workloads. The new solution is designed to operate within existing data centre setups, reducing the need for extensive infrastructure changes.

IRHX combines liquid cold plates for chip-level cooling with air-based fans to dissipate heat, enabling the system to be deployed alongside traditional air-cooled racks. This hybrid approach addresses the increasing energy and cooling requirements associated with running dense generative AI workloads.

Cooling designed for Blackwell GPU racks

The IRHX system was announced alongside AWS’s new EC2 P6e and P6e Ultra instances, which incorporate Nvidia’s Blackwell-based GB200 NVL72 racks. These racks include 72 GPUs interconnected with NVLink and provide 360 petaflops of FP8 compute and over 13 TB of memory.

The new cooling system allows AWS to support higher rack densities without relying exclusively on water cooling, which can be more difficult to manage and retrofit. AWS stated that IRHX maintains compatibility with its Nitro infrastructure, ensuring uptime and performance without structural overhaul.

Focus on scalability and operational stability

Dave Brown, AWS Vice President of Compute and Machine Learning Services, said the company required a system that was energy-efficient, scalable, and capable of supporting high GPU densities without compromising data centre reliability. Traditional methods, such as chilled water cooling, were found to be less adaptable for retrofitting across AWS’s large-scale infrastructure.

By integrating liquid cooling directly with GPU components and using in-row fans for broader heat dissipation, AWS has created a modular solution that supports the growing demand for generative AI without extensive redesign.