Tenstorrent’s Galaxy Blackhole AI servers escape the event horizon
Summary
Tenstorrent has announced the general availability of its Galaxy Blackhole AI compute platform, a RISC-V-based system featuring 32 Blackhole accelerators within a 6U chassis, priced at $110,000. Each Galaxy system offers 1 TB of GDDR6, 16 TB/s memory bandwidth, and 23 petaFLOPS of dense FP8 performance. The accelerators are interconnected via a 100 Tbps Ethernet mesh, which can be extended to form larger clusters, such as the $440,000 base Galaxy Supercluster with four systems. Tenstorrent claims significant software stack improvements, enabling a four-node Supercluster to process a 100,000-token DeepSeek V3 prompt in under four seconds and generate 720p video faster than real-time. The company also states that "Ninety percent of models from Hugging Face just run on Tenstorrent."
Key takeaway
For CTOs and VP of Engineering evaluating AI infrastructure, Tenstorrent's Galaxy Blackhole platform presents a compelling alternative to higher-priced solutions like Nvidia's DGX. Its scalable mesh architecture and claimed performance for large language models and video generation, coupled with a significantly lower entry cost, warrant consideration for your next-generation AI deployments. Investigate its real-world performance with your specific workloads and model types.
Key insights
Tenstorrent's Galaxy Blackhole offers a scalable, cost-effective AI compute platform with competitive performance for large models.
Principles
- RISC-V architecture enables specialized AI acceleration.
- Dense mesh networks enhance multi-node AI system scalability.
Method
Tenstorrent uses a Python-based programming interface for writing optimized kernels, facilitating the porting and performance tuning of new AI models on its hardware.
In practice
- Deploy Galaxy Blackhole for large language model inference.
- Utilize for high-throughput video generation tasks.
- Explore for cost-efficient AI infrastructure scaling.
Topics
- Tenstorrent Galaxy Blackhole
- Blackhole Accelerators
- RISC-V Architecture
- AI Server Systems
- Ethernet Mesh Interconnect
Best for: CTO, VP of Engineering/Data, AI Engineer, AI Architect, MLOps Engineer, Director of AI/ML
Related on AIssential
Editorial summary, takeaway, and curation by AIssential. Original article published by The Register: Enterprise Technology News and Analysis.