OpenAI’s Jalapeño Claims 1.5–1.9x More AI Work Per Watt, Faces 2027 Scale Test

OpenAI says its first custom inference chip can return model responses faster while delivering more work from each unit of power—a combination that could lower the cost of serving AI at scale if it holds up beyond its initial tests. The company presented Jalapeño’s first benchmark results at Hot Chips, framing the chip as the opening generation of a longer-term silicon platform.

On three public models, OpenAI reported 1.5 to 1.9 times more AI work per watt at peak throughput and 1.7 to 3.6 times lower end-to-end latency than its comparison systems. On the InferenceX benchmark, Jalapeño also produced more tokens per user and more throughput per kilowatt than the tested Nvidia Blackwell system.

That is a useful result for a company serving increasingly interactive models and agents, where delays can compound across many sequential steps. But it is not a claim that Jalapeño has already displaced conventional accelerators: OpenAI plans to keep widely deploying Nvidia and other partners’ hardware for both training and inference.

A benchmark result, not yet a fleet

The comparisons cover GPT-OSS 120B, DeepSeek R1 670B and Kimi K2.5 1T. OpenAI says it measured the systems at matched user experience across high-throughput and low-latency operating points, then normalized results using each accelerator’s published chip power rating. Jalapeño is rated at 700 watts and drew sustained power at or below 550 watts on the tested workloads.

Designing around the full request

Jalapeño’s approach is not simply to add more compute. OpenAI says prompt processing, known as prefill, is compute-intensive, while generating tokens one by one, known as decode, is more constrained by memory bandwidth. Moving information between chips can add another delay.

The company designed the chip, memory, networking and serving software together to reduce those handoffs. Its stated mechanism is to place model state—including the KV cache, which stores context used during generation—locally, while activating the necessary compute, memory and network resources for each stage. The architecture is meant to stay useful as the balance between prompt work and response generation changes.

The boundaries of the first tests

The initial evidence is narrower than a production verdict. The observed runs used selected nominal 8k/1k, single-turn workloads; they did not include longer-context, multi-turn AgentX tests. Those workloads can put more pressure on components such as routing and cache management, which means the current scores do not settle how Jalapeño will perform across realistic agentic serving patterns.

Timing is another constraint. Jalapeño remains at the engineering-sample stage, while newer Nvidia Vera Rubin systems were beginning to ship to customers. Richard Ho, OpenAI’s head of hardware, estimated a very small Jalapeño deployment at the end of 2026, followed by more significant deployment in 2027.

Software is part of the wager

OpenAI is also presenting Jalapeño as a software-development experiment. It says AI helped the team reach tapeout in nine months, and that Codex with GPT-Astra brought three open-weight models outside the original production plan to high performance within two months. New model families still require new kernels and model-specific work.

OpenAI plans to begin deploying Jalapeño in its compute infrastructure by the end of 2026, while it continues production qualification, software maturation and validation on more models. It says a second generation is deep in development and a third is taking shape. The immediate test is whether that roadmap turns a strong controlled benchmark into reliable, scalable inference capacity.