SGLang-ATOM: Bring ROCm-Native Acceleration to SGLang Serving

2026-07-08 · Source: AMD ROCm Blogs · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Cloud Computing & IT Infrastructure · Depth: Intermediate, medium

Summary

SGLang-ATOM, released on July 08, 2026, integrates ROCm-native Large Language Model (LLM) acceleration into SGLang serving workflows for AMD Instinct GPUs. It acts as a bridge within the ROCm LLM inference stack, connecting SGLang's serving experience with ATOM's optimized execution path. This is achieved through SGLang's model registration mechanism, allowing ATOM to register model wrappers and override selected implementations with ROCm-native execution without altering the SGLang source tree. The design separates developer-facing serving integration from hardware-facing optimization, enabling a staged adoption path. SGLang-ATOM offers ROCm-native optimization, reduces time-to-market, and provides a near-zero migration path for existing SGLang applications. Benchmark results are tracked on the ATOM dashboard for MI355X and MI308X systems, aiding deployment evaluation.

Key takeaway

For AI Engineers or ML teams deploying LLMs on AMD Instinct GPUs, SGLang-ATOM offers a direct path to ROCm-native acceleration without rebuilding your existing SGLang serving stack. You can integrate ATOM's optimized execution via a plugin, preserving your application logic and accelerating time-to-market. Utilize the ATOM benchmark dashboard to validate performance on MI355X and MI308X systems, ensuring efficient deployment with minimal migration effort.

Key insights

SGLang-ATOM integrates ROCm-native ATOM acceleration into SGLang serving via a model registration plugin for AMD Instinct GPUs.

Principles

• Separate serving integration from hardware optimization.
• Preserve ecosystem surface for faster adoption.
• Reduce manual performance engineering.

Method

SGLang-ATOM uses SGLang's model registration to import ATOM package modules, dynamically generating wrapper classes that delegate model creation and execution to ATOM's ROCm-native backend.

In practice

• Evaluate LLM performance on MI355X and MI308X via ATOM dashboard.
• Add new model support by registering ATOM-side model and appending architecture name.

Topics

• LLM Serving
• ROCm Acceleration
• AMD Instinct GPUs
• SGLang
• ATOM Inference Engine
• Benchmarking

Code references

• ROCm/ATOM
• sgl-project/sglang

Best for: MLOps Engineer, NLP Engineer, Machine Learning Engineer, AI Engineer, AI Architect

Related on AIssential

• ATOM: Unlocking Extreme AMD Instinct Inference with Software-Hardware Co-Optimization — ATOM is a ROCm-first, co-optimized inference engine for extreme LLM performance on AMD Instinct GPUs.
• Taking on CUDA With ROCm: ‘One Step After Another’ — AMD's ROCm software stack is rapidly maturing, leveraging open-source tools and community engagement to challenge Nvidia's CUDA dominance.
• JAX-AITER: Bringing AMD’s Optimized AI Kernels to JAX on ROCm™ — JAX-AITER bridges JAX with AMD's optimized AITER kernels for significant performance gains on ROCm GPUs.
• ChatNeuroSim: An LLM Agent Framework for Automated Compute-in-Memory Accelerator Deployment and Optimization — ChatNeuroSim automates CIM accelerator design and optimization using LLM agents and design space pruning for faster, more efficient exploration.
• TileFuse: A Fused Mixed-Precision Kernel Library for Efficient Quantized LLM Inference on AMD NPUs — TileFuse enables efficient AWQ-style quantized LLM inference on AMD XDNA2 NPUs through fused mixed-precision kernels.
• NVIDIA Releases Polar, a Token-Faithful Rollout Framework for GRPO Training Across Codex, Claude Code, and Qwen Code — Polar simplifies RL system integration by abstracting the agent harness via a model API proxy.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by AMD ROCm Blogs.