OpenAI and Broadcom launches Jalapeño, a custom chip built for AI inference

On June 24, OpenAI introduced its inaugural custom-built Intelligence Processor, named Jalapeño, in partnership with Broadcom. This innovative chip is specifically engineered for Large Language Model (LLM) inference, which involves generating outputs from models that have already undergone training. Jalapeño represents a pioneering series of AI accelerators that aim to enhance the speed, reliability, and accessibility of AI systems. OpenAI detailed in a recent blog post that Jalapeño was crafted from scratch, leveraging its expertise in LLMs alongside its envisioned AI roadmap, software, and the computational needs of products like ChatGPT. Collaboration with Broadcom and Celestica was crucial for the engineering, implementation, and infrastructure surrounding the chip. Notably, Jalapeño is versatile enough to operate not only on OpenAI’s models but also on other LLMs. Early iterations of the Jalapeño chip have reportedly excelled in running AI workloads within testing facilities, boasting superior performance-per-watt compared to existing alternatives. Broadcom’s President and CEO, Hock Tan, emphasized the significance of this development, stating, “This is just the beginning of a multi-generation roadmap. By co-developing our industry-leading silicon directly with OpenAI, we are enabling the deployment of gigawatt-scale data centres with Microsoft and other partners beginning in 2026.” OpenAI further elaborated on its comprehensive approach, indicating that the organization is not only focused on developing cutting-edge models but is also actively constructing the foundational infrastructure, including chip architecture, memory systems, and deployment systems. The rapid development of the Jalapeño chip is particularly noteworthy; it progressed from initial design to manufacturing in a mere nine months. This timeline could mark the fastest development cycle for a high-performance custom semiconductor (ASIC) in history. OpenAI pointed out that the AI models utilized by consumers are instrumental in informing the hardware development necessary for future AI systems, thus streamlining the chip development process. Designed to execute AI models post-training, the Jalapeño chip serves as a robust hardware platform for consumer-focused AI applications. It is specifically optimized for AI inference, allowing models to respond to user prompts on ChatGPT, generate code, or engage with AI agents. OpenAI is optimistic that Jalapeño will facilitate more efficient responses, lower hardware expenses, and support a greater number of users per chip.