OpenAI and Broadcom announce chip designed for LLM inference at scale

2026-06-24 · Source: AI - Ars Technica · Field: Technology & Digital — Artificial Intelligence & Machine Learning, AI Hardware, Cloud Computing & IT Infrastructure · Depth: Intermediate, quick

Summary

OpenAI and Broadcom have announced "Jalapeño," a new Application-Specific Integrated Circuit (ASIC) specifically engineered for large language model (LLM) inference in data centers. This chip, developed over nine months based on detailed insights from OpenAI's research and future model roadmap, represents the first generation in a planned long-term project. OpenAI claims early testing indicates "performance per watt substantially better than current state-of-the-art" compared to existing inference systems, with a full technical report expected soon. The initiative reflects OpenAI's strategy to achieve vertical integration, reduce reliance on external suppliers like Nvidia, and enhance efficiency amidst a global compute crunch. Broadcom is expanding its custom chip offerings for hyperscalers, and both companies anticipate deploying Jalapeño chips in data centers by the end of this year.

Key takeaway

For AI Architects and Directors of AI/ML planning future large language model deployments, you should closely monitor the performance reports for OpenAI and Broadcom's Jalapeño chip. This custom ASIC's promise of "substantially better performance per watt" suggests a significant shift in inference efficiency, potentially reducing your operational costs and compute dependency. Prepare to evaluate specialized hardware solutions as they become available, as vertical integration strategies will increasingly define competitive advantage in AI infrastructure.

Key insights

OpenAI and Broadcom developed Jalapeño, a custom ASIC for LLM inference, aiming for superior performance and vertical integration.

Principles

• Custom silicon optimizes LLM inference.
• Vertical integration enhances performance and efficiency.
• Strategic partnerships drive specialized hardware development.

In practice

• Deploy specialized ASICs for LLM inference.
• Evaluate custom silicon for performance gains.
• Consider vertical integration for AI infrastructure.

Topics

• OpenAI
• Broadcom
• Jalapeño ASIC
• LLM Inference
• Custom Silicon
• Data Center Infrastructure

Best for: Investor, CTO, VP of Engineering/Data, AI Hardware Engineer, AI Architect, Director of AI/ML

Related on AIssential

• OpenAI and Broadcom unveil "Jalapeño," a custom chip built for LLM inference — OpenAI and Broadcom's "Jalapeño" chip is a custom, full-stack hardware solution for efficient LLM inference, developed rapidly.
• OpenAI and Broadcom unveil LLM-optimized inference chip — OpenAI and Broadcom co-developed Jalapeño, a specialized LLM inference chip, to optimize AI infrastructure for efficiency and broader access.
• OpenAI unveils its first custom chip, built by Broadcom — OpenAI's custom chip, Jalapeño, signifies a strategic vertical integration to optimize AI inference costs and scale.
• OpenAI's spicy new custom AI chip — OpenAI's custom "Jalapeño" chip, co-built with Broadcom in nine months, uses AI for design, achieving superior inference efficiency.
• OpenAI reveals its first AI processor: Jalapeño — OpenAI's Jalapeño chip, an ASIC for AI inference, aims to reduce GPU reliance and boost LLM performance.
• Optimizing Local LLM Inference on Constrained Hardware — Bypassing high-level LLM wrappers and direct llama.cpp tuning doubles performance on constrained hardware.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by AI - Ars Technica.