dVoting: Fast Voting for dLLMs

2026-02-12 · Source: Computation and Language · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Natural Language Processing, Emerging Technologies & Innovation · Depth: Advanced, quick

Summary

dVoting is a novel, fast voting technique designed to enhance the reasoning capabilities of Diffusion Large Language Models (dLLMs) without requiring additional training. dLLMs represent a new paradigm that can generate tokens at arbitrary positions in parallel, offering significant potential for parallel test-time scaling. dVoting operates by iteratively refining model outputs: it samples multiple generations for a given prompt, identifies uncertain tokens through consistency analysis across samples, regenerates these specific tokens using a voting mechanism, and repeats the process until convergence. This method capitalizes on the observation that most token predictions are consistent, with performance hinging on a small subset of variable tokens. Evaluations show dVoting consistently improves performance, achieving gains of 6.22%-7.66% on GSM8K, 4.40%-7.20% on MATH500, 3.16%-14.84% on ARC-C, and 4.83%-5.74% on MMLU.

Key takeaway

For Research Scientists developing or deploying Diffusion Large Language Models, dVoting offers a significant, training-free method to improve reasoning performance. You should consider integrating dVoting into your dLLM inference pipelines to achieve substantial gains on benchmarks like GSM8K and MATH500, leveraging the inherent parallel generation capabilities of dLLMs for enhanced accuracy with acceptable computational overhead.

Key insights

dVoting improves dLLM reasoning by iteratively refining uncertain tokens through consistency analysis and voting.

Principles

• dLLMs enable parallel token generation.
• Performance hinges on a small subset of variable tokens.

Method

dVoting performs iterative refinement by sampling, identifying uncertain tokens via consistency analysis, regenerating them through voting, and repeating until convergence.

In practice

• Apply dVoting to boost dLLM reasoning.
• Utilize dLLMs for parallel test-time scaling.

Topics

• Diffusion Large Language Models
• dVoting
• Reasoning Enhancement
• Iterative Refinement
• Language Model Decoding

Code references

• fscdc/dVoting

Best for: Research Scientist, AI Researcher, AI Scientist, Deep Learning Engineer

Related on AIssential

• SAID: Accelerating Diffusion-Based Language Models via Scaffold-Aware Iterative Decoding — SAID accelerates DLLMs by prioritizing denoising on structural tokens and adaptively processing uncertain tokens.
• Learning from the Self-future: On-policy Self-distillation for dLLMs — d-OPSD adapts self-distillation for dLLMs by using suffix conditioning and step-level supervision, significantly boosting sample efficiency.
• When Does Delegation Beat Majority? A Delegation-Based Aggregator for Multi-Sample LLM Inference — Delegation-based aggregation using semantic entropy and reasoning geometry significantly improves LLM multi-sample inference over majority voting.
• OLLM: Options-based Large Language Models — OLLM uses a discrete latent space to model multiple next-token options, improving LLM control and performance.
• Diffusion-Proof: Recipe for Formal Theorem Proving Beyond Auto-Regressive Generation — Diffusion-Proof applies dLLMs to formal theorem proving, outperforming AR models by leveraging iterative denoising for long-range coherence.
• Diffusion LLMs from the Ground Up: Training, Inference, and Practical Engineering — dLLMs offer competitive performance and significantly faster inference than AR models, especially when initialized from pre-trained AR checkpoints.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Computation and Language.