OpenAI and Broadcom unveil LLM-optimized inference chip

2026-06-22 · Source: OpenAI News · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Cloud Computing & IT Infrastructure · Depth: Advanced, medium

Summary

OpenAI and Broadcom (NASDAQ: AVGO) unveiled Jalapeño on June 24, 2026, OpenAI's first Intelligence Processor designed specifically for LLM inference. This accelerator is the initial component in a multi-generation compute platform developed with Broadcom and Celestica, aiming to make advanced AI faster, more reliable, and more accessible. Jalapeño was architected from scratch based on OpenAI's deep understanding of LLM fundamentals, including its model roadmap and product needs. Early testing indicates it will deliver performance per watt substantially better than current state-of-the-art, achieved by optimizing data movement and balancing resources. The chip was developed from design to manufacturing tape-out in just nine months, partially accelerated by OpenAI's own models. It is slated for deployment at gigawatt scale with data center partners, beginning by the end of 2026.

Key takeaway

For Directors of AI/ML evaluating future infrastructure investments, OpenAI's Jalapeño chip signals a critical shift towards specialized hardware for LLM inference. You should prioritize solutions that offer superior performance per watt and are designed with full-stack optimization in mind. This approach can significantly reduce operational costs and enhance the reliability and speed of your interactive AI products, making advanced models more accessible and affordable for your users.

Key insights

OpenAI and Broadcom co-developed Jalapeño, a specialized LLM inference chip, to optimize AI infrastructure for efficiency and broader access.

Principles

• Full-stack control optimizes AI model performance.
• AI models can accelerate hardware design cycles.
• Specialized hardware improves LLM inference efficiency.

Method

Co-develop hardware and software, optimizing architecture for LLM kernels, memory, and networking to reduce data movement and balance resources.

In practice

• Design chips around specific LLM inference needs.
• Optimize for performance per watt in data centers.
• Integrate custom accelerators for interactive AI products.

Topics

• LLM Inference
• AI Accelerators
• Jalapeño Chip
• Broadcom
• Full-Stack AI
• Data Center Infrastructure
• Performance per Watt

Best for: Investor, CTO, VP of Engineering/Data, AI Hardware Engineer, AI Architect, Director of AI/ML

Related on AIssential

• OpenAI reveals its first AI processor: Jalapeño — OpenAI's Jalapeño chip, an ASIC for AI inference, aims to reduce GPU reliance and boost LLM performance.
• Optimizing Local LLM Inference on Constrained Hardware — Bypassing high-level LLM wrappers and direct llama.cpp tuning doubles performance on constrained hardware.
• OpenAI, Broadcom Develop AI Chip Called 'Jalapeno' — OpenAI and Broadcom are collaborating on a custom AI chip, "Jalapeno", signaling a strategic shift towards specialized hardware.
• Taalas Specializes to Extremes for Extraordinary Token Speed — Extreme specialization in AI chips by hardwiring models can yield superior performance and cost efficiency.
• A Practical Guide to CPU-Optimized LLM Deployment on Intel® Xeon® 6 Processors on AWS. — Intel Xeon 6 processors with vLLM enable cost-effective, high-throughput LLM inference on CPUs.
• OpenAI, Broadcom Unveil Chip to Run Models Faster, Cheaper — OpenAI and Broadcom unveiled a chip to run AI models faster and cheaper.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by OpenAI News.