GF-DiT: Scheduling Parallelism for Diffusion Transformer Serving

2026-06-11 · Source: Machine Learning · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Cloud Computing & IT Infrastructure · Depth: Expert, quick

Summary

GF-DiT is a policy-programmable runtime designed for elastic Diffusion Transformer (DiT) serving, addressing the inefficiencies of static parallelism in existing systems. Current DiT serving solutions assign fixed parallel configurations, which perform poorly due to the substantial heterogeneity across requests, execution stages, and system conditions, resulting in low GPU utilization and degraded service quality. GF-DiT dynamically adapts the parallelism of running requests based on workload demands and service objectives. It introduces an asynchronous execution abstraction that decomposes requests into independently schedulable trajectory tasks, enabling online GPU reallocation. Furthermore, GF-DiT proposes group-free collectives, a lightweight communication abstraction supporting low-overhead online formation and reconfiguration of execution groups. Implemented in vLLM-Omni, GF-DiT improves throughput by up to 6.01x, reduces mean latency by up to 95%, lowers SLO violation rates by up to 90%, and decreases communication-group setup overhead from 778 ms to approximately 60 µs compared to fixed-pipeline execution.

Key takeaway

For MLOps Engineers or AI Engineers deploying Diffusion Transformers, GF-DiT demonstrates a critical shift from static to dynamic parallelism. If you are struggling with poor GPU utilization or high latency in your current DiT serving infrastructure, consider implementing dynamic parallelism management. This approach can drastically improve throughput by up to 6.01x and reduce latency by up to 95%, ensuring better service quality and resource efficiency for your generative AI workloads.

Key insights

GF-DiT dynamically adapts GPU parallelism for Diffusion Transformers, significantly boosting performance and efficiency.

Principles

• GPU parallelism should be a first-class schedulable resource.
• Dynamic parallelism adaptation improves resource utilization.
• Decompose requests into independently schedulable tasks.

Method

GF-DiT uses an asynchronous execution abstraction to decompose requests into trajectory tasks, enabling online GPU reallocation. It also employs group-free collectives for low-overhead communication group reconfiguration.

In practice

• Implement dynamic parallelism for DiT serving.
• Utilize asynchronous execution for GPU reallocation.
• Adopt group-free collectives for communication efficiency.

Topics

• Diffusion Transformers
• GPU Scheduling
• Parallel Computing
• Elastic Serving
• vLLM-Omni
• Communication Collectives

Best for: Computer Vision Engineer, AI Scientist, Research Scientist, MLOps Engineer, Machine Learning Engineer, AI Engineer

Related on AIssential

• Deft Scheduling of Dynamic Cloud Workflows with Varying Deadlines via Mixture-of-Experts — DEFT uses a Mixture-of-Experts DRL architecture with a graph-adaptive gating mechanism to optimize dynamic cloud workflow scheduling.
• FLARE: Diffusion for Hybrid Language Model — FLARE systematically converts hybrid-attention LLMs to support both AR and diffusion decoding from a single checkpoint, prioritizing transfer data quality.
• AI's GPU problem is actually a data delivery problem — Inefficient data delivery, not GPUs, is the primary bottleneck for AI workload performance and scalability.
• Unlock Massive Token Throughput with GPU Fractioning in NVIDIA Run:ai — Fractional GPU allocation significantly boosts LLM inference capacity and efficiency without sacrificing latency.
• CoReDiT: Spatial Coherence-Guided Token Pruning and Reconstruction for Efficient Diffusion Transformers — CoReDiT efficiently prunes Diffusion Transformers by leveraging spatial coherence for token selection and reconstruction, significantly reducing computational cost.
• ORBIS: Output-Guided Token Reduction with Distribution-Aware Matching for Video Diffusion Acceleration — SW-HW co-design with output-guided, distribution-aware token reduction significantly accelerates video Diffusion Transformers by exploiting temporal consistency.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Machine Learning.