DeLS-Spec: Decoupled Long-Short Contexts for Parallel Speculative Drafting

2026-07-08 · Source: Computation and Language · Field: Technology & Digital — Artificial Intelligence & Machine Learning · Depth: Expert, quick

Summary

DeLS-Spec is a novel speculative decoding method designed to accelerate Large Language Model (LLM) inference by addressing limitations in existing block-parallel drafters. While methods like DFlash improve drafting efficiency by predicting entire blocks, they lack explicit intra-block causal conditioning. Previous attempts, such as Domino and DSpark, introduced causality but necessitated expensive training of draft models from scratch. DeLS-Spec overcomes this by decoupling long and short contexts, utilizing a fixed DFlash model as a long-context expert and integrating a lightweight, independently trainable local head as a short-context expert. This approach allows the local head to be trained with a standard next-token prediction objective, avoiding joint training and resulting in extremely low training costs. During inference, DeLS-Spec combines logits from both experts, offering modularity as the local head is not tied to a specific DFlash checkpoint. Experiments on Qwen3 models demonstrate that DeLS-Spec consistently enhances speedup and average acceptance length compared to DFlash across math, code, and dialogue benchmarks.

Key takeaway

For Machine Learning Engineers optimizing LLM inference, DeLS-Spec significantly boosts speedup and acceptance length. It achieves this without high training costs. If you use block-parallel drafters like DFlash, evaluate integrating DeLS-Spec's decoupled expert architecture. This method lets you utilize existing models while independently training a lightweight component. It streamlines deployment and improves performance across math, code, and dialogue benchmarks.

Key insights

Decoupling long-short contexts in speculative decoding with an independently trained local head significantly accelerates LLM inference with minimal training cost.

Principles

• Decouple long and short context experts.
• Train local components independently for flexibility.
• Combine expert logits for enhanced performance.

Method

DeLS-Spec employs a fixed DFlash model as a long-context expert and trains a lightweight local head as a short-context expert independently via next-token prediction. At inference, it combines long and short-context logits.

In practice

• Accelerate LLM inference on Qwen3 models.
• Improve speedup for math, code, dialogue.
• Reduce speculative decoding training cost.

Topics

• DeLS-Spec
• Speculative Decoding
• LLM Inference Acceleration
• Block-Parallel Drafting
• Qwen3 Models
• Training Cost Optimization

Best for: AI Scientist, Machine Learning Engineer, AI Engineer

Related on AIssential

• JetFlow: Breaking the Scaling Ceiling of Speculative Decoding with Parallel Tree Drafting — JetFlow breaks speculative decoding's scaling ceiling by combining one-forward drafting efficiency with branch-wise causal conditioning.
• Spec-AUF: Accept-Until-Fail Training under Train-Inference Misalignment for Masked Block Drafters — Spec-AUF aligns drafter training with inference by supervising only the accepted token prefix, improving speculative decoding efficiency.
• Train and Run DFlash Speculative Decoding — DFlash accelerates LLM inference by predicting and verifying entire blocks of future tokens in a single pass.
• Speculoos…No, Speculative Decoding: The Trick That Made My Old MacBook 3x Faster — Speculative decoding accelerates LLM inference by using a small draft model to predict tokens, verified by a larger model in parallel.
• 8x Faster Inference: DFlash with Block Diffusion Draft Trees — DDTree and IceCache offer distinct approaches to enhance LLM inference efficiency through improved speculative decoding and KV-cache management.
• FLy: A New Paradigm for Speculative Decoding — Accepting Semantically Correct Drafts Beyond Exact Match — FLy accelerates LLM inference by accepting semantically valid drafts, overcoming exact-match limitations without training.

See Counsel's argued verdicts on the open AI decisions leaders are weighing →

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Computation and Language.