Openai's Jalapeño chip sets new standards in AI inference speed and efficiency

OpenAI has announced that its first custom inference chip, Jalapeño, has demonstrated industry-leading performance in AI inference, delivering higher throughput and lower latency compared to existing hardware. The chip is capable of handling more AI work per unit of power while also providing faster response times. This is achieved through a single architecture that avoids the typical trade-off between speed and efficiency found in current systems. The results of Jalapeño's testing show significant improvements across multiple models, including GPT‑OSS 120B, DeepSeek R1, and Kimi K2.5 1T. On these models, Jalapeño delivered 1.5 to 1.9 times more AI work per watt at peak throughput and 1.7 to 3.6 times lower end-to-end latency than comparison systems. For highly interactive workloads, it achieved up to 4.1 times higher performance. Jalapeño is part of a broader full-stack advantage for OpenAI, which allows the company to design models, products, serving software, chips, memory, networking, and systems together. This integrated approach enables continuous improvements across all layers of the technology stack. The chip is also part of a multigenerational platform, with future generations in development. The performance of Jalapeño was evaluated using a matched user experience, measuring how much useful AI work each system can complete per unit of power while meeting the latency requirements for customers and interactive agents. This is particularly important for agents, which need to complete many steps in sequence, where delays can compound across an entire task. To test Jalapeño's performance in practice, OpenAI used the InferenceX benchmark from SemiAnalysis, which measures the full process of serving an AI request. Jalapeño was compared with leading commercially available AI systems across a range of operating conditions, from high-throughput serving to highly interactive, low-latency use. The results showed a better combination of throughput, power efficiency, and latency, placing Jalapeño on the Pareto frontier. Jalapeño's performance was also measured against other systems, such as the GB200 and GB300, with specific metrics including response time, tokens per second, and mixed performance per kilowatt. For example, on the GPT‑OSS 120B model, Jalapeño achieved 0.69 milliseconds versus 1.87 milliseconds for GB200, with 535 tokens per second versus 1,459 tokens per second. On the DeepSeek R1 model, Jalapeño delivered 1.43 milliseconds versus 5.90 milliseconds, with 700 tokens per second versus 169 tokens per second. For the Kimi K2.5 model, Jalapeño achieved 1.44 milliseconds versus 5.48 milliseconds, with 694 tokens per second versus 182 tokens per second. The development of Jalapeño was accelerated by AI itself, which enabled the team to move from initial design to tapeout in nine months. AI was used to explore implementations, shorten design, measurement, and verification loops, and continuously iterate on model workloads. It also helped optimize the chip’s arithmetic circuits, allowing the team to fit more compute performance into the chip on schedule. Jalapeño was designed as a clear, predictable programming target for both humans and AI. Engineers can describe work through local tensors, explicit communication, and predictable synchronization. AI can then optimize how that work is mapped, placed, scheduled, and coordinated across the system. This structure provides a tractable way for AI to tackle the traditionally difficult problem of parallel programming. The flexibility of Jalapeño's architecture was further demonstrated by the team's ability to bring three open-weight models to high performance within two months using Codex with GPT‑Astra. For selected GPT‑OSS attention and mixture-of-experts blocks, AI-generated implementations ran 1.5 to 1.8 times faster than human-expert-written ones. Jalapeño's efficiency and performance are expected to support broader adoption and continued investment in better models, products, and infrastructure. OpenAI plans to begin deploying Jalapeño within its compute infrastructure by the end of the year, with future generations in development. The results so far highlight the potential of designing the full system together to deliver more responsive, capable, and agentic AI more efficiently to more people.