SeLaR: Selective Latent Reasoning in Large Language Models

2026-04-09 · Source: Computation and Language · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Natural Language Processing · Depth: Expert, quick

Summary

SeLaR (Selective Latent Reasoning) is a new training-free framework designed to enhance reasoning in large language models (LLMs) by addressing limitations in existing Chain-of-Thought (CoT) and latent reasoning approaches. CoT's effectiveness is often limited by discrete token sampling, while prior latent reasoning methods, which use soft embeddings or hidden states, suffer from global activation perturbing high-confidence steps and soft embeddings collapsing too quickly. SeLaR introduces an entropy-gated mechanism that selectively activates soft embeddings only during low-confidence reasoning steps, maintaining discrete decoding for high-confidence steps to preserve stability. Furthermore, it employs an entropy-aware contrastive regularization to prevent soft embeddings from collapsing, encouraging broader exploration of alternative latent reasoning paths. Experiments across five reasoning benchmarks show SeLaR consistently outperforms standard CoT and other state-of-the-art training-free methods.

Key takeaway

For research scientists developing or deploying large language models, SeLaR offers a training-free method to improve reasoning performance. You should consider integrating SeLaR's selective latent reasoning to enhance model stability during high-confidence steps while still enabling robust exploration of alternative paths when the model is less certain. This approach can lead to more consistent and accurate reasoning outputs without requiring extensive retraining.

Key insights

SeLaR improves LLM reasoning by selectively applying soft embeddings and regularization to enhance exploration at low-confidence steps.

Principles

• Preserve discrete decoding for high-confidence steps.
• Activate soft embeddings only at low-confidence steps.
• Regularize soft embeddings to prevent collapse.

Method

SeLaR uses an entropy-gated mechanism for selective soft embedding activation and entropy-aware contrastive regularization to encourage exploration of latent reasoning paths.

In practice

• Apply soft embeddings only when confidence is low.
• Use contrastive regularization to diversify latent paths.

Topics

• Large Language Models
• Chain-of-Thought
• Latent Reasoning
• Soft Embeddings
• Entropy-Gated Mechanism

Best for: Research Scientist, AI Scientist, NLP Engineer, Machine Learning Engineer

Related on AIssential

• Tyler: Typed Latent Reasoning for Language Models -- When to Think, What to Compute, and How Much to Allocate — Tyler dynamically manages latent computation in LLMs, deciding when, what type, and how much to allocate for improved reasoning.
• Think in Sentences: Explicit Sentence Boundaries Enhance Language Model's Capabilities — Explicitly modeling sentence boundaries with delimiters enhances LLM reasoning by inducing structured, sentence-by-sentence processing.
• ReTreVal: Reasoning Tree with Validation and Cross-Problem Memory for Large Language Models — Combining structured exploration, iterative refinement, explicit validation, and persistent memory significantly enhances LLM multi-step reasoning.
• Context compression finally works in production: new research cuts LLM input 16x without the accuracy hit — Latent Context Language Models compress LLM input context before decoding, significantly boosting speed and memory efficiency without major accuracy loss.
• Entropy-Gated Latent Recursion — EGLR expands language model reasoning by adding a deterministic layer-recursion axis to stochastic sampling.
• The Evolution of Reasoning in Small Language Models with Yejin Choi - #761 — High-quality, diverse data and integrated learning approaches are key to enhancing small language model reasoning.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Computation and Language.