Evidence-State Rewards for Long-Context Reasoning

2026-07-02 · Source: Artificial Intelligence · Field: Technology & Digital — Artificial Intelligence & Machine Learning · Depth: Expert, quick

Summary

Maven is a reinforcement learning framework designed to enhance long-context reasoning in models by addressing the challenge of locating, revising, and synthesizing evidence across lengthy inputs. Unlike existing methods that reward only final answers or static evidence extraction, Maven introduces an editable evidence memory and provides action-level feedback on how intermediate actions alter the model's evidence state. It credits "add" actions by marginal gain and hindsight contribution, "link" actions by evidence synergy, and "drop" actions by improved answer support after removing misleading evidence. These rewards are assigned within GRPO. Maven significantly outperforms outcome-only RL and evidence-identification baselines on Llama and Qwen models across LongBench v2, LongReason, and RULER, yielding more sufficient evidence sets and reduced distractor retention.

Key takeaway

For Machine Learning Engineers developing long-context reasoning systems, you should shift focus from one-shot evidence extraction to optimizing stateful evidence navigation. Consider implementing action-level rewards that provide granular feedback on how evidence additions, links, and removals impact the model's evidence state. This approach, demonstrated by Maven, can lead to more robust evidence sets and significantly reduce the retention of distracting information, improving overall reasoning performance in your large language models.

Key insights

Optimizing stateful evidence navigation with action-level rewards significantly improves long-context reasoning in LLMs.

Principles

• Long-context RL benefits from stateful evidence navigation.
• Intermediate action feedback improves evidence processing.
• Reward mechanisms can target marginal gain, synergy, and support.

Method

Maven employs an editable evidence memory and defines an answer-conditioned evidence-state value. It rewards add actions by marginal gain/hindsight, link actions by synergy, and drop actions by improved answer support, assigned via GRPO.

In practice

• Implement action-level rewards for evidence manipulation.
• Integrate editable evidence memory into RL frameworks.
• Design rewards for evidence addition, linking, and removal.

Topics

• Reinforcement Learning
• Long-Context Reasoning
• Evidence Extraction
• Large Language Models
• GRPO
• Evidence-State Rewards

Best for: Research Scientist, AI Engineer, AI Scientist, Machine Learning Engineer, NLP Engineer

Related on AIssential

• Beyond Reward Engineering: A Data Recipe for Long-Context Reinforcement Learning — A diverse data recipe alone significantly improves long-context reasoning in reinforcement learning for LLMs.
• Reward Design for Physical Reasoning in Vision-Language Models — Reward design significantly shapes VLM physical reasoning, with accuracy-based rewards offering the strongest overall gains.
• Structured Role-Aware Policy Optimization for Multimodal Reasoning — SRPO improves LVLM multimodal reasoning by assigning role-aware, token-level credit for visual evidence and reasoning.
• From Verdict to Process: Agentic Reinforcement Learning for Multi-Stage Fact Verification — End-to-end agentic reinforcement learning with process-aware rewards optimizes multi-stage fact verification for better performance and efficiency.
• Reasoning Arena: Trace Tournaments When Verifiable Rewards Fall Short — Reasoning Arena transforms uninformative identical rewards into rich relative signals by comparing reasoning traces head-to-head.
• Feedback-Driven Tool-Use Improvements in Large Language Models via Automated Build Environments — Automated environment generation and verifiable rewards significantly boost LLM tool-use performance via reinforcement learning.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Artificial Intelligence.