NAND Reimagined in High-Bandwidth Flash to Complement HBM

2026-02-16 · Source: Big Data & AI News - EE Times · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Emerging Technologies & Innovation, Cloud Computing & IT Infrastructure · Depth: Intermediate, short

Summary

High-bandwidth flash (HBF) is an emerging memory solution designed to significantly boost capacity and performance for AI inference workloads by vertically stacking multiple layers of NAND dies. Unlike high-bandwidth memory (HBM), which stacks DRAM for bandwidth, HBF stacks 3D NAND arrays to enhance parallel I/O. SK Hynix recently presented a hybrid memory architecture combining eight HBM3E stacks with eight HBF stacks alongside an Nvidia Blackwell (B200) GPU, demonstrating a 2.69× improvement in performance per watt over HBM-only setups. HBF leverages HBM's vertical stacking concept and a parallel sub-array architecture for high-density integration and independent read/write channels, making it ideal for read-intensive inference tasks. SK Hynix and Sandisk anticipate trial versions and samples in 2026, with commercial products expected by early 2027.

Key takeaway

For Machine Learning Engineers designing AI inference systems, HBF offers a compelling solution to the memory wall by providing significantly higher capacity at lower cost than HBM. Consider integrating HBF into hybrid memory architectures, especially for read-intensive edge AI applications or as a capacity extension for HBM in data centers, to achieve substantial performance per watt improvements and store terabytes of model data.

Key insights

HBF stacks NAND dies for high-capacity, high-throughput memory, complementing HBM in AI inference architectures.

Principles

• HBF excels in read-intensive AI inference.
• Hybrid HBM-HBF architectures improve performance per watt.
• Vertical stacking enhances memory density and bandwidth.

Method

HBF combines 3D NAND flash with advanced packaging and interconnection, using a parallel sub-array architecture for independent read/write channels to achieve high throughput.

In practice

• Use HBF for large AI model storage.
• Deploy HBF in edge AI for pre-trained models.
• Pair HBF with HBM for capacity extension in data centers.

Topics

• High-bandwidth Flash
• AI Inference
• Hybrid Memory Architectures
• 3D NAND
• Memory Capacity

Best for: Machine Learning Engineer, NLP Engineer, Computer Vision Engineer, AI Architect, AI Engineer, MLOps Engineer

Related on AIssential

• Massive AI Storage Demand Creates a New Memory Wall — AI's massive memory demands necessitate new architectures like high-bandwidth flash, optimizing capacity and sequential bandwidth over traditional DRAM.
• [Server] The Bottleneck of Compute: How Humanity Weaves the Soul of Memory with Silicon — HBM overcomes the "Memory Wall" by prioritizing wide parallel data paths and 3D stacking over frequency.
• Unleashing Low-Bit Inference on Ascend NPUs: A Comprehensive Evaluation of HiFloat Formats — HiFloat formats, particularly HiF4, offer robust low-bit quantization for LLM inference on Ascend NPUs, especially at 4-bit precision.
• B200 to Vera Rubin: What NVIDIA Changed Again — NVIDIA's Vera Rubin platform significantly boosts AI compute and memory bandwidth, introducing specialized hardware for long-context inference.
• Rebellions Bets on Memory-Centric Architecture as It Weighs IPO Options — Memory-centric AI architectures and supply chain security are critical for large-scale LLM inference and market traction.
• SK hynix Ships Samples of 12-Layer Next-Gen ‘HBM4E’ — SK hynix's HBM4E samples demonstrate significant advancements in AI memory performance, power efficiency, and thermal management.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Big Data & AI News - EE Times.