REFACT: Adaptive Fact Restatement for Compact and Faithful Chain-of-Thought Reasoning

2026-07-23 · Source: Computation and Language · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Natural Language Processing · Depth: Expert, quick

Summary

REFACT is an adaptive fact-restatement citation framework designed to address large language models' tendency to drift from supplied context during long-form reasoning, particularly with sparse or conflicting evidence. Existing grounding methods often fail to ensure cited content sufficiently supports local inferences and final answers. REFACT trains models to determine when a reasoning step requires contextual grounding and the appropriate granularity for restating source facts, thereby preventing unsupported inferences and excessive fact copying. It optimizes this process through a two-stage SFT-to-RL pipeline, utilizing a citation-utility reward to ensure cited facts are well-formed, traceable, and sufficient for the answer. Experiments on LongBench, LV-Eval, and ConFiQA demonstrate that REFACT enhances long-context QA and counterfactual faithfulness while significantly reducing token consumption. The framework yields denser reasoning traces by preserving more answer-bearing evidence with fewer restated facts, published on 2026-07-23.

Key takeaway

For Machine Learning Engineers developing LLM applications requiring robust long-form reasoning, consider integrating REFACT to enhance contextual grounding. This framework improves long-context QA and counterfactual faithfulness by adaptively restating facts, preventing unsupported inferences. Your models will generate denser, more evidence-rich reasoning traces while substantially reducing token consumption, leading to more efficient and reliable outputs. Explore the provided code and data to implement this two-stage SFT-to-RL approach.

Key insights

REFACT grounds LLM reasoning by adaptively restating facts, ensuring faithfulness and compactness while reducing token use.

Principles

• Reasoning steps need adaptive contextual grounding.
• Citations can be answer-supporting intermediate states.
• Reward well-formed, traceable, answer-sufficient facts.

Method

REFACT employs a two-stage SFT-to-RL pipeline. It trains models to decide when and at what granularity to restate source facts, using a citation-utility reward for well-formed, traceable, and answer-sufficient facts.

In practice

• Improve long-context QA faithfulness.
• Reduce LLM reasoning token consumption.
• Generate denser, more evidence-rich traces.

Topics

• Large Language Models
• Chain-of-Thought Reasoning
• Fact Restatement
• Contextual Grounding
• Token Efficiency
• Reinforcement Learning

Code references

• NEUIR/REFACT

Best for: Research Scientist, AI Engineer, AI Scientist, Machine Learning Engineer, NLP Engineer

Related on AIssential

• REFACT: Adaptive Fact Restatement for Compact and Faithful Chain-of-Thought Reasoning — REFACT adaptively grounds LLM reasoning steps with context, ensuring faithful and compact outputs by integrating citations as intermediate states.
• REFACT: Adaptive Fact Restatement for Compact and Faithful Chain-of-Thought Reasoning — ReFact adaptively grounds LLM reasoning in context by selectively restating answer-sufficient facts, yielding compact and faithful traces.
• ReContext: Recursive Evidence Replay as LLM Harness for Long-Context Reasoning — RECONTEXT improves LLM long-context reasoning by replaying relevant evidence without training or external memory.
• ADaPT: Token-Level Decoupling for Efficient Large Reasoning Models — ADaPT decouples efficiency and correctness in large reasoning models via a token-level dual-process framework.
• ReTreVal: Reasoning Tree with Validation and Cross-Problem Memory for Large Language Models — Combining structured exploration, iterative refinement, explicit validation, and persistent memory significantly enhances LLM multi-step reasoning.
• Reliable Chain-of-Thought via Prefix Consistency — Truncating and regenerating LLM Chain-of-Thought traces reveals a reliability signal for improved answer aggregation.

See Counsel's argued verdicts on the open AI decisions leaders are weighing →

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Computation and Language.