Stop adding more GPUs: Weka's new storage platform reduces load by caching 100% of an AI model's pre-calculated tokens

2026-07-21 · Source: VentureBeat · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Cloud Computing & IT Infrastructure · Depth: Advanced, short

Summary

Weka has launched its NeuralMesh 6 software platform and Wekapod 3 hardware line, designed to reduce GPU memory load and inference costs in production AI. The platform introduces Augmented Memory Grid, which aggregates NAND flash to function like GPU memory at a lower cost, addressing the issue of repeated recomputation of pre-calculated tokens in long context windows. NeuralMesh 6 offers composable and virtual multi-tenancy, supporting over 1,000 tenants per cluster and up to 50,000 tenants with 50 composable clusters. It also provides unified file and object storage, claiming two orders of magnitude higher performance than conventional S3 for non-AWS GPU clouds. Other features include metadata-first replication for faster data availability and AlloyFlash, which mixes TLC and QLC NAND flash for optimized cost and performance, alongside Always-On data reduction. This aims to improve GPU utilization and accelerate AI workload deployment.

Key takeaway

For AI Architects and MLOps Engineers scaling large language model inference, you should evaluate Weka's NeuralMesh 6 and Wekapod 3 to mitigate escalating GPU memory costs. By caching 100% of pre-calculated tokens on cheaper flash storage, you can achieve significantly higher GPU utilization and faster deployment of new AI workloads. Consider its composable multi-tenancy and unified storage capabilities to optimize your infrastructure and reduce operational overhead, especially for multi-turn conversational AI or retrieval systems.

Key insights

Extending GPU memory with cheaper flash storage significantly reduces AI inference costs and improves utilization.

Principles

• Cache 100% of pre-calculated tokens to avoid recomputation.
• Mix TLC and QLC flash for optimal cost/performance.
• Unified file/object storage eliminates data duplication.

Method

NeuralMesh 6 uses Augmented Memory Grid to cache pre-calculated tokens on NAND flash, routing latency-sensitive tasks to TLC and bulk to QLC, accessible via unified file/object paths.

In practice

• Implement Augmented Memory Grid for long context AI models.
• Utilize composable multi-tenancy for isolated AI workloads.
• Adopt unified storage for training and inference data.

Topics

• AI Inference Optimization
• GPU Memory Extension
• Flash Storage
• NeuralMesh 6
• Augmented Memory Grid
• Multi-tenancy

Best for: CTO, VP of Engineering/Data, AI Engineer, AI Architect, MLOps Engineer, Director of AI/ML

Related on AIssential

• Overcoming the Memory Wall in Enterprise AI — AI inference is memory-bound, leading to idle GPUs and high costs, solvable with disaggregated memory solutions.
• You Can Just Download More Tokens/Sec — DeepSeek V4 Flash with DSpark offers superior speed and prefill, making local AI more productive than larger, slower models.
• Exclusive: Mindbeam touts dramatic performance improvements in CPU-based AI inference — CPU-based ternary LLM inference significantly boosts performance and reduces memory, complementing GPUs for diverse AI workloads.
• Lambda's MLPerf Inference v6.0: hardware leap, software maturity, research breakthrough — Hardware, software, and routing optimizations significantly boost AI inference performance and reduce latency.
• The Memory in Your Thumb Drive Could Fix AI’s Big Problem — High Bandwidth Flash (HBF) offers a cost-effective, high-capacity memory solution for read-heavy AI inference workloads.
• Taalas achieves 17000 tokens/second — Specialized silicon for AI inference can overcome GPU limitations, reducing cost and increasing speed.

See Counsel's argued verdicts on the open AI decisions leaders are weighing →

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by VentureBeat.