Majestic Labs Raises $100M for Memory Pooling AI Server

2026-05-28 · Source: Big Data & AI News - EE Times · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Cloud Computing & IT Infrastructure, Emerging Technologies & Innovation · Depth: Advanced, medium

Summary

AI chip startup Majestic Labs has secured \$100 million in Series A funding for its innovative memory-pooled server design, targeting AI inference workloads. Founded in 2023 by ex-Google and Meta silicon veterans, the company's architecture aims to provide up to 100 TB of DRAM per accelerator, significantly exceeding current HBM capabilities. Majestic Labs addresses the challenge of memory bandwidth-limited AI models by disaggregating memory from compute, enabling independent scaling. Their system uses two custom silicon dies—a memory interface chiplet and a many-core AI acceleration chip—alongside standard LPDDR memory. This design simplifies programming by presenting a single, contiguous memory space and offers flexible compute-to-memory ratios, scaling from one to 12 compute chips and 8 to 128 TB of memory. The company plans to tape out its chips this year, with servers shipping to lead customers in 2027, promising substantial cost and power advantages for hyperscalers and large enterprises.

Key takeaway

For AI Architects evaluating infrastructure for large-scale AI inference, Majestic Labs' memory-pooled servers present a significant shift. Their disaggregated memory approach offers up to 100 TB of DRAM per accelerator. This directly addresses memory bandwidth limitations and simplifies programming. You should consider this architecture to reduce operational costs and power consumption. It optimizes compute-to-memory ratios, especially for memory-intensive models. Plan to assess their 2027 server shipments for future deployment strategies.

Key insights

Majestic Labs' memory-pooled server disaggregates memory from compute via a high-bandwidth interface, providing vast, unified DRAM for AI inference.

Principles

• AI inference is memory bandwidth-limited.
• Disaggregate memory from compute for scaling.
• Simplify programming with flat memory space.

Method

Majestic Labs employs a memory interface chiplet and AI accelerator chip, aggregating standard LPDDR via a proprietary high-speed interface. This creates a single, contiguous memory space for compute chips, simplifying AI workload optimization.

In practice

• Configure servers for prefill or decode.
• Target hyperscalers and neoclouds.
• Reduce GPU over-specification costs.

Topics

• Memory Pooling
• AI Inference Servers
• Disaggregated Memory
• LPDDR DRAM
• AI Accelerators
• Data Center Infrastructure

Best for: Investor, AI Architect, AI Hardware Engineer, Director of AI/ML

Related on AIssential

• This chip startup just raised $135M on a bet that AI’s biggest bottleneck isn’t compute — it’s memory — AI inference is increasingly a memory scaling problem, not solely a compute challenge.
• Startup Boosts Scale-Up to 1000+ GPUs in a Single Domain — Disaggregated server design and software enable resilient, large-scale GPU inference clusters with flexible topologies.
• Taalas Specializes to Extremes for Extraordinary Token Speed — Extreme specialization in AI chips by hardwiring models can yield superior performance and cost efficiency.
• The RTX 5080 Is 3x Faster Than NVIDIA’s $3,999 “AI Supercomputer” — And It Doesn’t Matter — Memory capacity, not raw inference speed, defines the DGX Spark's primary value proposition.
• Breaking the Memory Wall in the Age of Inference — Integrating compute directly into SRAM cells via Digital In-Memory Compute (DIMC) effectively addresses AI inference memory bottlenecks.
• [AINews] H100 prices are melting *UP* — GPU rental prices are surging, advanced AI models are scaling, and local inference capabilities are rapidly improving.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Big Data & AI News - EE Times.