OpenAI Unveils Jalapeño Inference Chip: InferenceX Benchmarks Show 1.9x Better Efficiency Than NVIDIA GB300

Global AI giant OpenAI officially released the latest benchmark data for its in-house custom ASIC inference chip "Jalapeño" on August 25, 2026, shaking up an AI hardware market long dominated by NVIDIA. Developed in close partnership with communications semiconductor giant Broadcom, the dedicated chip marks OpenAI's official move from being purely an algorithm and model developer into low-level hardware architecture design, aiming to use a specialized architecture to fully solve its ever-growing compute costs and energy bottlenecks.

This breakthrough targets the toughest pain points in bringing generative AI to commercial use: inference latency and operating energy use. As load on global cloud servers surges and new real-time AI agents spread, the marginal benefit of traditional general-purpose GPUs in high-concurrency scenarios is gradually hitting a ceiling, and the arrival of Jalapeño is a bombshell for the entire digital tech industry.

From an industry strategy point of view, OpenAI's in-house chip strategy is not only about making its software services respond faster but also a key move to gain independence in the global race for computing resources. Today's tech giants are all racing to build their own compute ecosystems. Jalapeño's test results not only confirm the overwhelming advantage of custom ASICs for inference but also signal that AI computing architecture is about to undergo a new paradigm shift.

Key takeaways

• Efficiency beyond flagships: 1.5 to 1.9 times more compute per watt than NVIDIA's GB200/GB300.
• Much lower latency: end-to-end latency drops by 1.7 to 3.6 times, greatly shortening time between tokens (TBT).
• Broad model support: validated on the InferenceX platform with support for GPT-OSS 120B, DeepSeek R1 and Kimi K2.5 1T.
• A powerful partnership: built jointly by OpenAI and Broadcom.

A technical deep dive: the efficiency and latency breakthroughs revealed by InferenceX benchmarks

The Jalapeño chip shows overwhelming hardware optimization results in inference efficiency. According to authoritative data from the independent evaluation platform InferenceX, when running very large frontier models such as GPT-OSS 120B, DeepSeek R1 and the trillion-parameter Kimi K2.5 1T, Jalapeño delivers 1.5 to 1.9 times better AI performance per watt than NVIDIA's current flagship GB200 and GB300 architectures. This means that with the same data center power supply and cooling limits, cloud operators can deploy nearly twice the compute throughput and greatly cut the energy needed to generate each million tokens.

A big cut in end-to-end latency removes the experience bottleneck for real-time interactive applications. Test data shows that Jalapeño's end-to-end latency across complex workloads is 1.7 to 3.6 times lower than the comparison systems, with a breakthrough on the key metric of time between tokens (TBT). As OpenAI VP of Hardware Richard Ho emphasized, the chip strikes a perfect balance between ultra-low latency and very high throughput, ensuring next-generation algorithms can deliver smooth, near-millisecond feedback for high-concurrency real-time voice, dynamic screen interactions and complex workflow orchestration.

Reshaping the market and business landscape: ending dependence on a single architecture and optimizing inference costs

Mass production of dedicated ASIC chips signals a redistribution of supply chain power toward top AI giants. For years, model developers like OpenAI have been heavily constrained by NVIDIA's high hardware prices and production allocations, and the capital expenditure (CapEx) and operating expenses (OpEx) of inference infrastructure have become the biggest test of profitability. Through its strategic alliance with Broadcom, OpenAI can not only precisely customize the instruction set for its own private software stack but also avoid the chip area general-purpose GPUs waste, achieving maximum economic efficiency at the level of its core information processing architecture.

Lower inference costs will directly accelerate the widespread adoption of enterprise AI applications. As model parameter counts keep growing, the network bandwidth and server resources consumed during inference have long exceeded those of initial training, and high API prices have held back digital transformation at small and mid-sized businesses. Jalapeño's outstanding efficiency makes it possible to deploy autonomous agents capable of making their own decisions at scale, and is expected to drive explosive growth in high-value business scenarios such as real-time financial risk control, multimodal customer service systems and collaborative edge computing.

Strategic opportunities and supply chain effects for Taiwan's semiconductor ecosystem

Jalapeño's successful development brings real benefits to Taiwan's semiconductor manufacturing and advanced packaging clusters. Although OpenAI and Broadcom led the chip's architecture and logic design, the foundry processes and high-bandwidth memory (HBM) advanced packaging needed for high-performance computing are still rooted in Taiwan's world-class supply chain. The technological moat of Taiwan's leading foundry and advanced packaging companies in 2.5D/3D heterogeneous integration is the foundation that makes mass production of custom high-performance ASICs like this possible.

Global server manufacturers and cooling module suppliers will also benefit from the purchasing wave for the new architecture. To fit Jalapeño's new high-density inference modules, Taiwanese server manufacturers and major liquid cooling component makers are quickly adjusting their product lines to meet next-generation data centers' custom standards for energy-efficient racks. This hardware upgrade trend, shifting from pure GPUs to hybrid GPU and dedicated ASIC architectures, will bring long-term growth momentum to Taiwan's hardware ecosystem.