The year everyone quotes DeepSeek V4 Flash’s “98GB” number, nobody quotes which GPU it actually ran…

2026-07-19 · Source: Machine Learning on Medium · Field: Technology & Digital — Artificial Intelligence & Machine Learning · Depth: Intermediate, quick

Summary

DeepSeek V4 Flash, a 284B parameter model with a 1 million token context window released in April 2026, is often cited as running on "98GB" with a single budget GPU. However, this "98GB" figure applies only to a compressed version, with actual hardware requirements for real workloads falling into the 96-128GB range. While the model is genuinely good, claims of it running on a \$400 consumer card are unsubstantiated. Real progress in local inference for V4 Flash is occurring on prosumer hardware. For instance, KTransformers, an inference framework, achieved over 20 tokens per second decode on a single RTX 5090, a \$2,000 card. Additionally, an experimental llama.cpp fork by antirez enables V4 Flash on CPU and Apple Metal backends using 128GB of system RAM. This indicates that while frontier models are becoming accessible on high-end workstations, they are not yet viable on entry-level consumer GPUs.

Key takeaway

For AI Engineers evaluating DeepSeek V4 Flash for local deployment, understand that the "98GB" figure is misleading. You should size your hardware for 96-128GB of VRAM or system RAM, not a budget GPU. If you aim to run frontier models locally, prioritize investing in a high-end workstation with an RTX 5090 or 128GB+ RAM, and verify performance claims with actual token/second benchmarks before purchasing hardware based on rumors.

Key insights

The "98GB" DeepSeek V4 Flash claim misrepresents actual prosumer hardware needs for local inference.

Principles

• Model quantization numbers often understate full inference requirements.
• "Consumer hardware" for frontier models typically means prosumer rigs.
• Verify hardware claims with specific benchmarks and token/second rates.

Method

KTransformers supports V4-Flash for offloaded MoE serving, achieving 20+ tok/s on an RTX 5090. Antirez's llama.cpp fork runs it on CPU/Metal with 128GB RAM.

In practice

• Size hardware for 96-128GB for V4 Flash workloads.
• Try KTransformers on RTX 5090 for V4 Flash.
• Use antirez's fork on 128GB+ RAM workstations.

Topics

• DeepSeek V4 Flash
• Large Language Models
• Local Inference
• GPU Memory
• Model Quantization
• KTransformers
• RTX 5090

Best for: NLP Engineer, Machine Learning Engineer, AI Engineer, MLOps Engineer

Related on AIssential

• DeepSeek V4 - almost on the frontier, a fraction of the price — DeepSeek V4 models offer competitive performance and industry-leading low costs through significant efficiency gains.
• A few words on DS4 — The combination of powerful local models and efficient quantization enables frontier-level local AI experiences.
• [AINews] DeepSeek V4 Pro (1.6T-A49B) and Flash (284B-A13B), Base and Instruct — runnable on Huawei Ascend chips — DeepSeek V4 advances open-weight long-context and agentic coding through novel attention mechanisms and efficient architecture.
• I Ran Every Local LLM on a 24GB GPU So You Don’t Have To — Consumer GPUs like the RTX 3090 now enable practical local LLM inference, balancing cost, speed, and quality for various use cases.
• Introducing DeepSeek V4 Flash and V4 Pro in Microsoft Foundry — Optimal AI system design balances model quality, speed, and cost by orchestrating multiple specialized models.
• ChinAI #356: DeepSeek as Road Builder [修路人] — DeepSeek V4 achieves near-frontier AI capabilities with significant compute efficiency, particularly for long-context inference.

See Counsel's argued verdicts on the open AI decisions leaders are weighing →

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Machine Learning on Medium.