OpenAI and Broadcom announce chip designed for LLM inference at scale

In a groundbreaking collaboration, OpenAI, renowned for its innovations like ChatGPT and Codex, has partnered with Broadcom, a key player in the semiconductor industry, to introduce a new chip dubbed Jalapeño. This state-of-the-art chip is specifically engineered for large language model (LLM) inference, aiming to enhance performance in data centers. The Jalapeño chip represents the first step in a long-term initiative, with both companies emphasizing that it is merely the beginning of a series of advancements to come. Broadcom has stated that this Application-Specific Integrated Circuit (ASIC) was meticulously crafted from the ground up, drawing on extensive insights gained through dialogues with OpenAI’s researchers. The development process took a swift nine months, reflecting an agile approach to technology development. One of the key advantages of Jalapeño lies in its specialization for LLM inference, which promises to outpace the current systems deployed in existing data centers. According to OpenAI, initial tests indicate that Jalapeño significantly improves performance per watt compared to the current leading technologies. However, OpenAI has cautioned that performance assessments are still ongoing, and a comprehensive technical report detailing the findings is expected in the upcoming months.