OpenAI unveils its first custom chip, built by Broadcom

On Wednesday, OpenAI introduced its inaugural custom-designed inference processor, developed in partnership with Broadcom. This new chip, dubbed Jalapeño, is tailored specifically to meet the unique demands of OpenAI’s inference systems. The creation of Jalapeño was aided by OpenAI's own AI models, which guided its development. While the chip is currently undergoing testing, early indications suggest it delivers a significantly improved performance-per-watt compared to existing leading alternatives. This collaboration was officially unveiled in October, although speculation regarding OpenAI's chip initiatives has circulated for some time. The move is seen as a strategic effort to lessen the company’s reliance on Nvidia’s GPUs. Similar to Google and Amazon, which have also developed custom chips known as "AI accelerators" to enhance machine learning performance, OpenAI's approach aims to optimize efficiency. OpenAI’s president, Greg Brockman, shared insights into the company's chip development strategy during an episode of its in-house podcast following the announcement of the Broadcom partnership. He emphasized their comprehensive understanding of workload requirements and the importance of creating solutions that specifically address underserved areas. Jalapeño is purpose-built for inference, which involves executing pre-trained AI models in response to user inputs. In their announcement, OpenAI highlighted the chip's cost-effectiveness, especially in running real-time coding models. Although it appears that more resource-intensive tasks like pre-training will likely continue to depend on Nvidia hardware, even minor reductions in inference costs could significantly benefit the company's financial health. The optimization of inference systems is expected to play a pivotal role in the future economics of AI. OpenAI's endeavors extend beyond model development; they are also constructing data centers and creating agentic products like Codex. The introduction of custom chips is a crucial step in enhancing their technological ecosystem. OpenAI stated, "We are not only developing cutting-edge models or building products on top of them; we are designing the infrastructure that supports them: chip architecture, kernels, memory systems, networking, scheduling, deployment systems, and user experiences." By operating across various levels of the tech stack, OpenAI aims to streamline each component towards a unified goal: to enhance the speed, reliability, and affordability of its AI models for users.