ChinAI #361: What can CANN do for China's independent compute capacity?

2022-03-07 · Source: ChinAI Newsletter · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Cloud Computing & IT Infrastructure, Emerging Technologies & Innovation · Depth: Advanced, medium

Summary

DeepSeek V4 recently demonstrated "chip-model synergy" with Huawei's Ascend chips, a capability previously exclusive to Nvidia's CUDA ecosystem. This achievement highlights the maturation of Huawei's Compute Architecture for Neural Networks (CANN), which is transitioning from an "infancy" to a "youth" phase, enabling developers to independently resolve issues and contribute code. Early adopters like Shanghai-based AIGCode, which used Ascend due to 2024 GPU shortages, initially faced a four-month support wait but later achieved a 65% Model Flops Utilization (MFU) during pre-training a 7B-scale Mixture-of-Experts model, nearly doubling the industry average. Huawei open-sourced CANN's core code last December, and its developer communities now exceed 4 million members, supporting over 70 major AI models. This progress addresses critical dimensions like adaptation efficiency, performance ceilings, and production-grade reliability, as evidenced by a leading bank integrating AI into core risk management and contributing 34 optimizations to vLLM-Ascend.

Key takeaway

For AI Architects evaluating compute infrastructure for large-scale model deployment, this development signals a viable alternative to Nvidia's CUDA. You should investigate Huawei's CANN and Ascend ecosystem, especially given its demonstrated 65% MFU and growing developer community exceeding 4 million members. Consider piloting projects on Ascend chips to assess adaptation efficiency and production-grade reliability for your specific workloads, potentially reducing reliance on a single vendor.

Key insights

Huawei's CANN is maturing, fostering an independent chip-model synergy ecosystem challenging Nvidia's CUDA dominance.

Principles

• Ecosystem maturity enables developer independence.
• Open-sourcing drives community contributions.
• MFU measures chip compute utilization.

In practice

• Migrate HPC solvers to Kunpeng CPUs.
• Optimize vLLM for Ascend NPUs.
• Integrate AI into core risk management.

Topics

• Huawei Ascend
• CANN (Compute Architecture for Neural Networks)
• NVIDIA CUDA
• Chip-Model Synergy
• Open-Source Ecosystems
• Model Flops Utilization
• DeepSeek V4

Best for: Investor, CTO, VP of Engineering/Data, AI Engineer, AI Architect, Director of AI/ML

Related on AIssential

• NVIDIA drops DGX Station for Windows (1-Trillion Parameter desktop). Who else is ready to run LLaMA-Behemoth locally? — Aggressive quantization remains crucial for local inference of trillion-parameter models, even with powerful new hardware.
• Flush With Cash From OpenAI, Opal Is Making an AI-Powered Audio Gadget — Opal, funded by OpenAI, is expanding from webcams into AI-powered audio hardware.
• Microsoft Build: MAI-Thinking-1 and MAI Family models, Surface RTX Spark Dev Box, and OpenClaw in Windows — Microsoft is integrating its AI stack from custom silicon to frontier models and agent-native OS, emphasizing transparency and clean data.
• Extreme Low-Bit Inference in Reasoning Models: Failure Modes and Targeted Recovery — Aggressive 2-bit quantization in LRMs causes specific generation pathologies, not just accuracy drops, which can be mitigated.
• Accelerate LLM model loading and increase context windows with GPUDirect on Amazon FSx for Lustre and TurboQuant — Direct storage-to-GPU data paths and KV cache compression drastically cut LLM load times and expand context windows.
• Unbox one of NVIDIA's first co-packaged optics switches with us. See why we bet on CPO early. — Co-packaged optics (CPO) significantly improves power efficiency and reliability in large-scale GPU cluster networking.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by ChinAI Newsletter.