On August 25, 2026, OpenAI released benchmark results for Jalapeño, its first in-house-designed inference chip. Using InferenceX with three publicly available models, the chip processed 1.5 to 1.9 times more work per watt at peak than Nvidia's GB200 or GB300, and completed responses in 1.7 to 3.6 times less time. This puts concrete numbers behind the "significantly better power efficiency than current state-of-the-art" claim OpenAI made when it first unveiled the chip in June.
The fact that a first-generation custom ASIC performed this well carries real weight. However, the measurements were taken under fixed-length inference conditions—nominally 8,000 input tokens and 1,000 output tokens—and the comparison did not include Nvidia's next-generation Rubin chip. Under what conditions does Jalapeño's advantage hold, and how far can it reshape OpenAI's compute infrastructure?
1.5x to 1.9x Power Efficiency Across Three Models
InferenceX, the benchmark OpenAI used, is a public tool that maps the relationship between inference server throughput and user-perceived speed along a continuous curve. Handling a large volume of requests at high parallelism typically slows down per-user response times. OpenAI states that Jalapeño pushed out the "Pareto frontier"—maximizing throughput while maintaining a given response speed—across all three models tested.
| Model | Compared Against | Peak mixed TPS/kW | E2E Latency | Minimum TBT |
|---|---|---|---|---|
| GPT-OSS 120B | GB200 | 85,448 vs. 44,960 (~1.9x) | 1.03s vs. 1.80s | 0.69ms vs. 1.87ms |
| DeepSeek R1 670B | GB300 | 19,641 vs. 11,781 (~1.7x) | 1.65s vs. 5.99s | 1.43ms vs. 5.90ms |
| Kimi K2.5 1T | GB300 | 18,195 vs. 11,862 (~1.5x) | 1.56s vs. 5.31s | 1.44ms vs. 5.48ms |
Mixed TPS refers to combined input and output tokens processed per second, while TBT is the time between tokens during generation. This is not a table showing peak efficiency and minimum latency measured at the same operating point. Even so, for DeepSeek R1, minimum TBT shrank from 5.90ms to 1.43ms, and per-user generation speed rose from 169 to 700 tokens per second. Jalapeño outperformed its comparison targets on both peak efficiency and minimum TBT.
Co-Design That Reduces KV Cache Movement
LLM inference relies heavily on compute during the "prefill" phase, when input is read all at once, and tends to be bottlenecked by memory bandwidth during "decode," when answers are generated one token at a time. For massive Mixture of Experts (MoE) models, the time spent moving necessary weights and KV caches between chips is also non-trivial. GPUs can handle a wide range of computations, but that versatility comes with a complex memory hierarchy and the burden of maintaining software compatibility.
Jalapeño was designed primarily for LLM inference, placing model state close to where it's used. Compute cores, memory, and inter-chip networking were built as an integrated whole, allowing the same accelerator cluster to be reallocated even as the ratio of prefill to decode workload changes. According to OpenAI, reducing KV cache movement and communication wait times made it easier to extract peak compute performance even with small batch sizes.
Generative AI was also used to bring up the software stack. OpenAI says it went from initial design to tape-out in nine months, and optimized three open-weight models—not part of the original product plan—for high performance within two months. For certain attention and MoE blocks in GPT-OSS, AI-generated implementations ran 1.5 to 1.8 times faster than existing implementations written by human experts. However, this figure applies to selected compute blocks, not evidence that the entire model sped up by the same ratio.
What the Fixed 8K/1K Test Didn't Measure
The denominator in the published graphs is not total measured facility power. OpenAI normalized the figures using each accelerator's rated package TDP: 700W for Jalapeño, 1,200W for GB200, and 1,400W for GB300. Jalapeño's sustained power draw under this workload was below 550W, yet the calculation used 700W. The figures are not tokens/MW inclusive of host CPUs, in-rack networking, cooling, or power conversion, nor do they reflect total cost of ownership including purchase price.
The workload itself was also a single-turn test with a nominal 8K input and 1K output, using single-token prediction rather than multi-token prediction, which finalizes multiple predicted tokens at once. In real-world AI agents, context grows with every conversational turn, processing pauses during tool calls, and stored KV caches get reused repeatedly. The fixed-length test does not measure long-running operation, including routing, cache management, or recovery from failures.
SemiAnalysis visited OpenAI's lab and visually confirmed the InferenceX runs—valuable as independent verification. At the same time, the figures were supplied by OpenAI, which explicitly states it has not run the full InferenceX test suite, nor AgentX, which replays long-context, multi-turn workloads. Nvidia, too, describes the fixed 8K/1K setup as a controlled single-turn workload, and currently promotes AgentX as the comparison that more closely resembles real-world deployment. Jalapeño's next round of testing will need to close this gap with reproducible measurements under matching conditions.
Not Yet a Case for Replacing Rubin or CUDA
In OpenAI's published table, Jalapeño went head-to-head only with GB200 and GB300—not Rubin. The claim that Jalapeño "also beat Rubin" comes from SemiAnalysis placing this Jalapeño data alongside separately published Vera Rubin results. The models, input lengths, inference methods, and software maturity all differ; this was not a simultaneous, apples-to-apples measurement.
On August 24, Nvidia announced that Vera Rubin NVL72, running DeepSeek V4 Pro 1.6T on AgentX, achieved up to 30 times the throughput per MW compared to GB300, at 160 tokens/second/user. This too is an early-stage measurement from Nvidia, and SemiAnalysis has not yet reviewed it. Directly comparing Jalapeño's 8K/1K figures with Rubin's AgentX figures as if they were on the same scale would mean adding up scores from two different competitions.
The same caution applies to claims about CUDA. The fact that OpenAI got three large models running quickly and tuned them down to low latency shows the company can stand up a new software stack rapidly for its own purposes. But it's not evidence that this replaces the vast ecosystem of libraries, training and HPC workloads, or compatibility for external developers that CUDA supports. OpenAI itself has stated it will continue to deploy accelerators from Nvidia and others widely for both training and inference.
The Gap Between a Year-End Rollout and the 10GW Plan
Jalapeño is currently an engineering sample undergoing production qualification and software maturation. OpenAI plans to begin deploying it within its own compute infrastructure by the end of 2026, with Gen 2 already in full development and the direction for Gen 3 taking shape. Broadcom is handling chip implementation and networking, while Celestica manages everything from boards to full rack integration.
Behind this lies the multi-year 10GW plan that OpenAI and Broadcom announced in October 2025. Deployment of the target racks is scheduled to begin in the second half of 2026, with completion targeted by the end of 2029. However, the 10GW figure spans multiple generations of custom accelerators and networking combined—it does not mean the first-generation Jalapeño alone will be running at 10GW scale by year's end. At the time of the announcement, the disclosed contractual status for rack deployment was the signing of a term sheet.
SemiAnalysis reports that, following the measurements taken on the A0 revision, a B0 revision improving performance-per-watt by roughly 25% is now in production, with volume manufacturing gradually ramping up in 2027. The configuration reportedly combines an ASIC rack housing 128 Jalapeño units with a host rack, together drawing about 160kW. If such systems can move past the initial year-end rollout into stable, sustained operation—and reproduce their advantage under AgentX and full-facility power measurements—OpenAI will have a realistic path toward shifting power-constrained inference workloads onto its own custom silicon.