China’s LongCat-2.0 is a 1.6T Model Trained Without NVIDIA

2026-07-08 · Source: Towards AI - Medium · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Cloud Computing & IT Infrastructure · Depth: Advanced, quick

Summary

Meituan, a Chinese food-delivery company, announced LongCat-2.0 on June 30, 2026, a 1.6-trillion-parameter language model trained entirely on approximately 50,000 domestic AI chips without NVIDIA GPUs. This achievement is attributed to the model's innovative Mixture-of-Experts (MoE) design, which activates only about 48 billion parameters, or 3%, per token. This sparse activation dramatically reduces the per-step computational workload and, more importantly, the network traffic. This optimization allows the model to be trained effectively on a cluster of second-tier accelerators, overcoming the typical interconnect bottlenecks that arise when using hardware less powerful than NVIDIA's Hopper architecture. The design incorporates specific tricks like zero-computation experts and a cross-layer shortcut.

Key takeaway

For AI Architects and ML Engineers facing hardware constraints or seeking alternatives to NVIDIA, LongCat-2.0 demonstrates a viable path. Your teams should investigate specialized Mixture-of-Experts designs, particularly those optimizing per-step network traffic, to make frontier-scale model training feasible on domestically sourced or less powerful accelerators. This approach can significantly reduce hardware requirements and training costs.

Key insights

A specialized Mixture-of-Experts design enables training frontier-scale models on less powerful hardware by minimizing per-step network traffic.

Principles

• Interconnect, not computation, often bottlenecks large model training.
• Sparse MoE designs can reduce per-step resource demands.
• Overlap FFN computation with network communication.

Method

LongCat-2.0 uses a Shortcut-connected MoE (ScMoE) to overlap FFN computation with network communication, and Multi-Teacher On-Policy Distillation (MOPD) for post-training.

In practice

• Train large models on non-NVIDIA accelerators.
• Implement sparse MoE architectures for efficiency.

Topics

• LongCat-2.0
• Mixture-of-Experts
• AI Accelerators
• Model Training
• Hardware Constraints
• Meituan
• ScMoE

Best for: MLOps Engineer, Research Scientist, Investor, AI Scientist, Machine Learning Engineer, AI Architect

Related on AIssential

• Accelerating Mixture-of-Experts Execution with FarSkip-Collective Models — FarSkip-Collective accelerates MoE models by overlapping computation and communication, eliminating idle GPU time during distributed execution.
• Meituan's LongCat-2.0 shows China can train massive AI models without Nvidia — China has demonstrated the ability to train trillion-parameter AI models using only domestic hardware, challenging US export controls.
• MIT Proved 90% of Every AI Model Is Dead Weight. It Took 8 Years for the Hardware to Catch Up. — Most neural network weights are redundant; efficient subnetworks can be trained directly with modern hardware and tooling.
• CuTeGen: An LLM-Based Agentic Framework for Generation and Optimization of High-Performance GPU Kernels using CuTe — CuTeGen iteratively refines GPU kernels using CuTe abstraction and delayed profiling, achieving competitive performance.
• Meituan open sources LongCat-2.0, the 1.6T, near-frontier agentic coding model that's been leading OpenRouter — trained entirely on Chinese chips — Meituan's LongCat-2.0 demonstrates near-frontier AI can be developed on domestic Chinese ASICs with competitive performance and cost.
• Latest open artifacts (#17): NVIDIA, Arcee, Minimax, DeepSeek, Z.ai and others close an eventful year on a high note — The open model ecosystem is rapidly evolving with diverse architectures, data transparency, and specialized performance.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Towards AI - Medium.