What OpenAI Custom Inference Silicon Changes, What Remains Unaltered, and How to Audit Efficiency Ratios
A technical analysis of the OpenAI and Broadcom Jalapeño inference processor, 1.5x to 1.9x throughput-per-watt benchmark conditions, TSMC N3P silicon limits, and enterprise AI accelerator evaluation.
Japanese Edition: Read the original Japanese article on Substack
Video Episode (Japanese Audio): Watch on YouTube
Executive Summary: Targeted Latency and Efficiency vs. Shared Silicon Supply Realities
On June 24, 2026, OpenAI and Broadcom Inc. announced Jalapeño, a custom application-specific integrated circuit (ASIC) engineered exclusively for large language model inference. According to official joint announcements, the processor aims to unify the high raw throughput of mainstream accelerators with the ultra-low interactive latency of dedicated inference engines. Initial system deployments within production clusters were scheduled to commence by the end of 2026. However, final production performance metrics remain under active characterization, and the inaugural June announcement contained zero absolute performance numbers.
On August 25, 2026, OpenAI published preliminary benchmark results evaluated against NVIDIA Blackwell-generation hardware (specifically the GB200 and GB300). Technology publications reported that Jalapeño delivered:
- Between 1.5x and 1.9x higher maximum token throughput per watt.
- Between 1.7x and 3.6x lower end-to-end request latency.
Crucially, these efficiency multipliers are governed by strict measurement conditions:
- Mathematical Denominator: Ratios were calculated using manufacturer rated thermal design power (TDP)—700W for Jalapeño, 1,200W for GB200, and 1,400W for GB300—rather than real-world dynamic power draw measured at the utility meter.
- Workload Boundary: Evaluations comprised isolated single-turn prompts with 8,000 input tokens and 1,000 output tokens.
- Algorithmic Constraints: Tests operated without speculative decoding.
- Data Provenance: All telemetry originated from OpenAI internal benchmarks utilizing the SemiAnalysis InferenceX test suite; while analysts witnessed runs in OpenAI laboratories, the research firm did not independently execute the benchmark suite.
Equally significant is what custom silicon does not alter. According to technical reporting from SemiAnalysis, Jalapeño compute logic is fabricated on TSMC N3P process node—the identical fabrication node utilized for NVIDIA next-generation Rubin GPUs. Furthermore, Jalapeño incorporates fourth-generation High Bandwidth Memory (HBM4), identical to Rubin. Consequently, custom ASIC development does not bypass global semiconductor bottlenecks; OpenAI must queue for the exact same foundry allocation and advanced packaging lines as commercial merchant silicon vendors.
For corporate infrastructure architects, chief technology officers, and procurement leads evaluating enterprise AI infrastructure, this analysis provides three critical analytical frameworks:
- Auditing the denominator of claimed efficiency ratios (rated TDP vs. empirical dynamic power vs. facility-level utility draw).
- Deconstructing benchmark configurations (prompt lengths, token ratios, generation algorithms, and hardware generational parity).
- Verifying benchmark provenance (vendor self-reporting vs. independent third-party replication).

Methodological Boundaries: Documented Records, Author Calculations, and Interpretation
To maintain analytical rigor, this evaluation segments information into three categories:
- Documented Records: Explicit statements, engineering specifications, and benchmark disclosures published in official corporate press releases, engineering whitepapers, SEC filings, and accredited industry research.
- Author Calculations: Mathematical ratios, percentages, and conversions calculated directly from published figures.
- Author Interpretation: Synthesized evaluations and frameworks regarding enterprise procurement, explicitly designated as analytical deductions.
Primary source documentation comprises the OpenAI-Broadcom joint announcement (June 24, 2026), OpenAI preliminary results disclosure (August 25, 2026), SemiAnalysis architectural teardowns, accredited technology reporting (WinBuzzer and Igor Lab), and official datasheets from NVIDIA, Google Cloud, and Amazon Web Services, retrieved and cross-verified on November 2, 2026.

Technical Foundations: The Mechanics of Transformer Inference
Evaluating Jalapeño requires examining the dual-phase nature of autoregressive model execution:
- Prefill Phase: The model processes the input context prompt in parallel. This phase is heavily compute-bound, saturating matrix multiplication tensor cores.
- Decode Phase: The model generates output tokens sequentially. Each token generation step requires streaming parameter weights and historical key-value (KV) attention cache states from off-chip memory into on-chip registers. This phase is heavily memory-bandwidth-bound.
- Speculative Decoding: An optimization technique where a smaller draft model predicts multiple candidate tokens, which the larger target model verifies in a single forward pass, shifting memory-bound decoding toward compute saturation.
- Thermal Design Power (TDP): The maximum theoretical power dissipation specified by hardware vendors, distinct from real-world dynamic consumption.
- High Bandwidth Memory (HBM): Vertically stacked DRAM dies connected to logic through silicon interposers, progressing from HBM3E to HBM4.

Section 1: Officially Confirmed Parameters: Architecture, Roles, and Delivery Horizons
Official corporate announcements published on June 24, 2026, confirm several key operational parameters:
- Designation and Positioning: Jalapeño is OpenAI first custom “Intelligence Processor,” architected specifically for generative model inference. The companies framed the chip as the inaugural silicon release in a multi-generational computing partnership.
- Architectural Philosophy: The design was created from a blank slate tailored specifically to OpenAI proprietary model weights, software kernels, and serving systems, rejecting the general-purpose compromises of traditional GPUs.
- Corporate Division of Labor:
- OpenAI: Architecture definition, logic design, and software integration.
- Broadcom: Physical implementation, co-packaged optics, PCIe switching, and Tomahawk network interconnects.
- Celestica: System-level hardware engineering, PCB fabrication, and server rack assembly.
- Silicon Status: OpenAI stated that laboratory prototypes were operational at production clock frequencies and thermal targets, successfully executing production models including GPT-5.3-Codex-Spark.
- Development Velocity: The companies claimed a 9-month timeline from architecture freeze to tapeout. SemiAnalysis noted that initial team hiring and engineering commenced in mid-2024, representing approximately 16 months from initial staffing to tapeout.
- Deployment Trajectory: Initial cluster deployment was slated for late 2026, with Broadcom leadership noting multi-gigawatt infrastructure collaboration with Microsoft. SemiAnalysis projected that production volume would ramp progressively throughout 2027, with the bulk of deployments landing in late 2027.
- Omitted Public Data: The official announcement omitted fabrication process names, memory configurations, and numerical throughput data.

Section 2: Architectural Focus: Why Inference Demands Memory Bandwidth
In standard GPU clusters, memory bandwidth constraints during autoregressive decoding leave compute engines idling while waiting for weights and KV cache transfers.
Jalapeño addresses this through targeted dataflow optimization. OpenAI joint statement notes that the architecture minimizes data movement, balancing compute, memory, and networking to operate near theoretical hardware limits.
SemiAnalysis architectural reporting describes a partitioned layout:
- The monolithic die is partitioned into localized compute and memory zones, allowing execution units to access adjacent HBM dies with minimal interconnect latency.
- The system rejects disaggregated prefill-decode clustering. In disaggregated architectures, separate server pools handle input processing and token generation. SemiAnalysis explains that because input-to-output token ratios fluctuate continuously across real-world enterprise traffic, static pooling causes chronic underutilization and queue stalls. Jalapeño unifies prefill and decode across identical execution engines.
Industry peers are pursuing complementary optimizations. NVIDIA Vera Rubin NVL72 product disclosures combine Rubin GPUs with Groq LPX racks to target trillion-parameter models across million-token contexts.

Section 3: Deconstructing Reported Benchmarks: Conditions Behind the Multipliers
On August 25, 2026, OpenAI released preliminary benchmarks evaluated on the SemiAnalysis InferenceX test harness across three open-weights models: GPT-OSS 120B, DeepSeek R1 670B, and Kimi K2.5 1T. All tests evaluated single-turn interactions with 8,000 input tokens and 1,000 output tokens.
Reported performance differentials against NVIDIA Blackwell hardware were published as follows:
| Evaluated Model | Comparison Baseline | Reported Throughput Per Watt Gain | Reported Latency Reduction | Absolute Throughput (Tokens/s/kW) | Absolute End-to-End Latency |
|---|---|---|---|---|---|
| GPT-OSS 120B | NVIDIA GB200 NVL72 | ~1.9x higher | ~1.7x lower latency | 85,448 (Jalapeño) vs. 44,960 (GB200) | 1.03s (Jalapeño) vs. 1.80s (GB200) |
| DeepSeek R1 670B | NVIDIA GB300 NVL72 | ~1.7x higher | ~3.6x lower latency | Not disclosed in summary | 1.65s (Jalapeño) vs. 5.99s (GB300) |
| Kimi K2.5 1T | NVIDIA GB300 NVL72 | ~1.5x higher | ~3.4x lower latency | Not disclosed in summary | Not disclosed in summary |
Four technical conditions qualify these numbers:
Condition 1: Rated Thermal Design Power Denominator
The reported throughput-per-watt ratios were calculated by dividing raw token output by the rated maximum thermal design power (TDP) of each accelerator package:
- OpenAI Jalapeño: 700W rated TDP.
- NVIDIA GB200: 1,200W rated TDP.
- NVIDIA GB300: 1,400W rated TDP.
Author calculation shows the arithmetic impact of rated TDP denominators: 1,200W divided by 700W equals 1.71x, while 1,400W divided by 700W equals 2.0x. Baseline power ratings alone account for the majority of the reported multiplier. While OpenAI reported that Jalapeño sustained dynamic power remained below 550W during testing, corresponding dynamic measurements for GB200 and GB300 were not disclosed.
Condition 2: Absence of Speculative Decoding
All benchmarks evaluated sequential token-by-token decoding without speculative decoding. Because speculative decoding increases compute intensity, it disproportionately enhances GPU throughput.
Condition 3: Fixed Synthetic Prompt Ratios
The test suite evaluated exclusively an 8,000/1,000 token single-turn profile. Results from the AgentX benchmark, which simulates multi-turn conversational agents with sprawling KV caches, were not available.
Condition 4: Generational Alignment
SemiAnalysis observed that comparing Jalapeño to Blackwell is architecturally mismatched. Because Jalapeño utilizes HBM4 memory, its true peer is NVIDIA Rubin. SemiAnalysis estimates that on facility-level power (all-in utility MW), Jalapeño exceeds Rubin throughput per watt, but total cost per token is roughly comparable between the two architectures. Furthermore, these tests evaluated early A0 stepping silicon; a refined B0 revision currently in fabrication is projected to improve energy efficiency by an additional 25%.

Section 4: Industry Landscape: Memory Bandwidth and Fabrication Parity
Comparing publicly verifiable hardware specifications across leading AI accelerators illustrates architectural positioning:
| Accelerator Platform | Memory Capacity | Memory Technology | Memory Bandwidth | Evaluation Unit | Source Citation |
|---|---|---|---|---|---|
| NVIDIA Vera Rubin | 288GB | HBM4 | 19.2 TB/s | Per discrete GPU | NVIDIA official product page |
| NVIDIA GB200 NVL72 | 13.4TB | HBM3E | 576 TB/s rack total (~8.0 TB/s per GPU) | 72-GPU rack system | NVIDIA official product page |
| NVIDIA GB300 NVL72 | 20.0TB | HBM3E | Up to 576 TB/s rack total (~8.0 TB/s per GPU) | 72-GPU rack system | NVIDIA official product page |
| Google Cloud TPU7x | 192GiB | HBM | ~7.37 TB/s | Per discrete chip | Google Cloud documentation |
| Amazon Trainium3 | 144GB | HBM3e | 4.9 TB/s | Per discrete chip | AWS product specification |
| OpenAI Jalapeño | Unspecified | HBM4 | 15.4 TB/s | Per multi-chip package | SemiAnalysis technical teardown |
Jalapeño memory bandwidth of 15.4 TB/s per package places it near NVIDIA Rubin (19.2 TB/s). However, both chips depend on TSMC N3P lithography and HBM4 packaging lines. While custom ASICs optimize internal dataflow, they remain bound to the global semiconductor supply chain.
In rack deployment, Jalapeño clusters utilize 128 accelerators per server rack, co-developed with Celestica. Separately, Foxconn and OpenAI announced an agreement on November 21, 2025, to co-design and manufacture next-generation AI data center racks in United States facilities. However, that agreement covers generalized infrastructure and does not mention Jalapeño.
Furthermore, technology reporting indicates that OpenAI has explicitly reaffirmed its commitment to purchasing and deploying commercial merchant accelerators, including NVIDIA GPUs, across training and inference workloads. Jalapeño represents architectural diversification rather than total vendor replacement.

Section 5: Analytical Interpretation: What Changes and What Remains Constant
Synthesizing public disclosures highlights critical strategic takeaways:
- Efficiency Is Multiplicative: A reported 1.9x throughput-per-watt advantage is the product of silicon architecture multiplied by rated TDP baselines. Altering the denominator to actual facility-level power reshapes comparative efficiency.
- Latency Gains Are Real but Conditional: Single-turn prompt benchmarks prove that co-designing memory buses with proprietary model kernels dramatically compresses response times. Whether these gains persist under multi-turn agent workloads remains unverified.
- Merchant GPUs Remain Essential: Custom ASICs cannot instantly supplant merchant GPUs due to packaging constraints and software ecosystem maturity.
- Unverified Variables: Pricing economics, long-context conversational performance (AgentX), and wafer allocation volumes remain undisclosed.

Executive Implementation Framework: Auditing Accelerator Performance Claims
Enterprise technical evaluators reviewing vendor performance metrics should enforce a five-step audit protocol:
- Audit the Power Denominator: Determine whether efficiency claims divide by rated TDP, dynamic chip draw, or total facility utility power (including cooling and power distribution losses).
- Examine Algorithmic Conditions: Verify whether benchmarks utilized speculative decoding, structured sparsity, or low-bit quantization formats.
- Inspect Workload Topology: Check input context and generation lengths. Single-turn tests with short outputs conceal KV-cache eviction bottlenecks.
- Demand Generational Parity: Ensure new silicon is compared against contemporary architectures rather than prior-generation baselines.
- Verify Data Provenance: Ascertain whether benchmarks represent vendor self-reporting or independently reproduced runs.
Frequently Asked Questions
Q: Is Jalapeño fundamentally faster than NVIDIA GPUs? A: OpenAI benchmarks demonstrated lower latency and higher throughput per rated watt against Blackwell GB200 and GB300 under strict single-turn, 8,000/1,000-token conditions. However, against Rubin hardware utilizing speculative decoding, comparative cost per token is estimated to be roughly equivalent.
Q: Will OpenAI cease procuring NVIDIA accelerators? A: No. OpenAI has explicitly stated that it will continue large-scale procurement of NVIDIA GPUs for both model training and operational inference.
Q: Is Foxconn manufacturing Jalapeño server racks? A: Official documentation does not link Foxconn to Jalapeño. Foxconn partnered with OpenAI for general AI rack designs, whereas Celestica is the engineering partner for Jalapeño server systems.
Q: Why do development timelines vary between 9 and 16 months? A: OpenAI measured the 9-month window from formal architecture tapeout authorization, whereas SemiAnalysis tracked the 16-month window from initial team recruitment.
Primary Reference Documentation
- S1: OpenAI and Broadcom Joint Announcement (June 24, 2026): https://www.stocktitan.net/news/AVGO/open-ai-and-broadcom-unveil-llm-optimized-intelligence-jqpk7vkxf7jd.html
- S2: SemiAnalysis Architectural Analysis (August 25, 2026): https://newsletter.semianalysis.com/i/212647373/software
- S3: WinBuzzer Technology Report (August 26, 2026): https://winbuzzer.com/2026/08/26/openai-jalapeno-faster-inference-gb200-gb300-company-tests-xcxwbn/
- S4: Igor Lab Benchmarking Analysis: https://www.igorslab.de/en/openai-jalapeno-beats-nvidia-gb200-gb300-in-initial-inference-benchmarks-up-to-36x-lower-latency/
- S5: NVIDIA Vera Rubin NVL72 Specifications: https://www.nvidia.com/en-us/data-center/vera-rubin-nvl72/
- S6: NVIDIA GB200 NVL72 Specifications: https://www.nvidia.com/en-us/data-center/gb200-nvl72/
- S7: NVIDIA GB300 NVL72 Specifications: https://www.nvidia.com/en-us/data-center/gb300-nvl72/
- S8: NVIDIA Technical Blog (Attention Co-Design for Interactive Inference): https://developer.nvidia.com/blog/co-designing-ai-model-attention-for-fast-interactive-long-context-inference/
- S9: Google Cloud TPU7x Architecture Documentation: https://cloud.google.com/tpu/docs/tpu7x
- S10: Amazon Trainium3 Product Specifications: https://aws.amazon.com/ai/machine-learning/trainium/
- S11: Foxconn and OpenAI Partnership Announcement (November 21, 2025): https://www.foxconn.com/en-us/press-center/press-releases/latest-news/1905
- P1: OpenAI Official Announcement (June 24, 2026): https://openai.com/index/openai-broadcom-jalapeno-inference-chip/
- P2: OpenAI Preliminary Results Blog (August 25, 2026): https://openai.com/index/jalapeno-first-results/
- P3: Broadcom Official News Release (June 24, 2026): https://investors.broadcom.com/news-releases/news-release-details/openai-and-broadcom-unveil-llm-optimized-intelligence-processor