Amazon Web Services is building equipment to cool Nvidia GPUs as AI boom accelerates

Amazon has announced a significant development from its cloud division, unveiling new hardware specifically designed to cool the next generation of Nvidia graphics processing units (GPUs) utilized for artificial intelligence (AI) tasks. The surge in generative AI has led to a dramatic increase in energy demands for these GPUs, necessitating enhanced cooling solutions for companies that utilize them. Initially, Amazon considered establishing data centers capable of implementing widespread liquid cooling systems to effectively manage the heat generated by these power-intensive Nvidia processors. However, Dave Brown, Vice President of Compute and Machine Learning Services at Amazon Web Services (AWS), explained in a recent YouTube video that this approach would be impractical due to the time required for setup and the inefficiency of existing commercial equipment. Brown highlighted concerns about the physical space requirements of such systems and the potential for significantly increased water consumption. "They would take up too much data center floor space or increase water usage substantially," he noted. Instead, Amazon's engineering team developed the In-Row Heat Exchanger (IRHX), a versatile cooling solution that can be easily integrated into both existing and new data centers. While traditional air cooling methods sufficed for earlier Nvidia models, the latest offerings demand more robust cooling capabilities. With the introduction of this advanced cooling system, customers can now access AWS computing instances named P6e, which are optimized for the latest Nvidia technology. The GPUs in question, specifically the Nvidia GB200 NVL72, feature a single rack housing 72 interconnected Blackwell GPUs, enabling the training and operation of large-scale AI models. AWS, recognized as the leading provider of cloud infrastructure, has a history of developing its own hardware solutions. This includes custom chips for both general-purpose and AI computing, as well as proprietary storage servers and networking routers. By leveraging in-house hardware, Amazon minimizes reliance on third-party suppliers, positively impacting its financial performance. In the first quarter of this year, AWS reported its highest operating margin since at least 2014, contributing significantly to Amazon's overall net income. Meanwhile, Microsoft, the second-largest cloud services provider, is also making advances in chip development, recently unveiling its own cooling systems designed for its Maia AI chips. This innovative approach from AWS not only underscores its commitment to supporting the growing demands of AI but also reinforces its position at the forefront of cloud technology.