REFACT: Adaptive Fact Restatement for Compact and Faithful Chain-of-Thought Reasoning

2026-07-23 · Source: Takara TLDR - Daily AI Papers · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Natural Language Processing · Depth: Expert, quick

Summary

REFACT is an adaptive fact-restatement citation framework designed to enhance the faithfulness and compactness of large language models' long-form reasoning. It addresses the issue of reasoning traces drifting from context due to sparse, noisy, or conflicting evidence, a common problem with existing grounding methods that often fail to ensure cited content sufficiency. REFACT trains models to intelligently determine when and at what granularity a reasoning step requires contextual grounding, transforming citations into answer-supporting intermediate states. The framework is optimized via a two-stage SFT-to-RL pipeline, utilizing a citation-utility reward to ensure cited facts are well-formed, source-traceable, and sufficient for the answer. Experiments on LongBench, LV-Eval, and ConFiQA demonstrate REFACT's ability to improve long-context QA and counterfactual faithfulness while significantly reducing token consumption and producing denser, more evidence-rich reasoning traces.

Key takeaway

For Machine Learning Engineers developing LLM applications requiring robust long-form reasoning, REFACT offers a significant advancement. You should consider integrating this adaptive fact-restatement framework to mitigate reasoning drift and improve contextual grounding. This approach enhances both the faithfulness of your models' outputs and reduces token consumption, yielding denser, more reliable reasoning traces for complex tasks like long-context QA and counterfactual reasoning.

Key insights

REFACT adaptively grounds LLM reasoning steps with context, ensuring faithful and compact outputs by integrating citations as intermediate states.

Principles

• Adaptive grounding prevents reasoning drift.
• Citations function as intermediate states.
• Reward-based optimization for fact utility.

Method

REFACT employs a two-stage SFT-to-RL pipeline. A citation-utility reward function encourages well-formed, source-traceable, and answer-sufficient facts, guiding models to adaptively restate context for grounding reasoning steps.

In practice

• Improve long-context QA performance.
• Enhance counterfactual faithfulness.
• Reduce LLM token consumption.

Topics

• REFACT Framework
• Large Language Models
• Chain-of-Thought Reasoning
• Contextual Grounding
• Reinforcement Learning
• Token Efficiency

Code references

• NEUIR/REFACT

Best for: Research Scientist, AI Engineer, AI Scientist, Machine Learning Engineer, NLP Engineer

Related on AIssential

• REFACT: Adaptive Fact Restatement for Compact and Faithful Chain-of-Thought Reasoning — REFACT grounds LLM reasoning by adaptively restating facts, ensuring faithfulness and compactness while reducing token use.
• REFACT: Adaptive Fact Restatement for Compact and Faithful Chain-of-Thought Reasoning — ReFact adaptively grounds LLM reasoning in context by selectively restating answer-sufficient facts, yielding compact and faithful traces.
• Copy Less, Ground More: Overcoming Repetitive Copying in Long-Context Reasoning via Evidence-Aware Reinforcement Learning — Insufficient grounding causes repetitive copying in long-context LLMs, addressable by evidence-aware reward shaping.
• ReTreVal: Reasoning Tree with Validation and Cross-Problem Memory for Large Language Models — Combining structured exploration, iterative refinement, explicit validation, and persistent memory significantly enhances LLM multi-step reasoning.
• Teaching Reasoning Models When to Stop: Data, Rewards, and Self-Aware Decoding — Efficient CoT reasoning requires balancing length reduction with accuracy, often by leveraging model confidence.
• Reinforced Fast Weights with Next-Sequence Prediction — REFINE improves fast weight models for long-context tasks by shifting from next-token to next-sequence prediction via reinforcement learning.

See Counsel's argued verdicts on the open AI decisions leaders are weighing →

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Takara TLDR - Daily AI Papers.