REFACT: Adaptive Fact Restatement for Compact and Faithful Chain-of-Thought Reasoning

2026-07-25 · Source: cs.CL updates on arXiv.org · Field: Technology & Digital — Artificial Intelligence & Machine Learning · Depth: Expert, extended

Summary

ReFact is an adaptive fact-restatement citation framework designed to enhance large language models' (LLMs) long-form reasoning by ensuring contextual grounding and preventing reasoning drift. It trains models to selectively determine when a reasoning step requires evidence and at what granularity to restate source facts, avoiding both unsupported inferences and indiscriminate fact copying. Optimized via a two-stage Supervised Fine-Tuning (SFT) to Reinforcement Learning (RL) pipeline, ReFact uses a citation-utility reward to encourage well-formed, source-traceable, and answer-sufficient cited facts. Experiments on LongBench, LV-Eval, and ConFiQA demonstrate that ReFact improves long-context QA and counterfactual faithfulness, reducing average reasoning length by approximately 50% on LongBench v1 for Qwen3-4B and 8B models, while producing denser, more answer-sufficient reasoning traces.

Key takeaway

For Machine Learning Engineers developing LLMs for long-context reasoning, ReFact offers a robust approach to improve faithfulness and efficiency. You should consider implementing adaptive fact-restatement frameworks to ensure reasoning steps are contextually grounded and avoid relying on conflicting parametric knowledge. This method yields more compact and accurate traces, reducing token consumption while maintaining answer-bearing evidence.

Key insights

ReFact adaptively grounds LLM reasoning in context by selectively restating answer-sufficient facts, yielding compact and faithful traces.

Principles

• Grounding reasoning steps in context prevents drift.
• Citation utility optimizes for answer-sufficient evidence.
• Adaptive fact restatement reduces token verbosity.

Method

ReFact uses a two-stage SFT-to-RL pipeline. SFT establishes citation format, then RL optimizes a policy with a reward combining format, accuracy, traceability, and answerability criteria.

In practice

• Use a verifier model to ensure cited content sufficiency.
• Reward answerability over raw citation count.
• Integrate evidence indices for SFT supervision.

Topics

• Chain-of-Thought Reasoning
• Fact Restatement
• Long-context QA
• Reinforcement Learning
• LLM Faithfulness
• Token Efficiency

Code references

• NEUIR/REFACT

Best for: AI Engineer, Research Scientist, AI Scientist, Machine Learning Engineer, NLP Engineer

Related on AIssential

• REFACT: Adaptive Fact Restatement for Compact and Faithful Chain-of-Thought Reasoning — REFACT grounds LLM reasoning by adaptively restating facts, ensuring faithfulness and compactness while reducing token use.
• REFACT: Adaptive Fact Restatement for Compact and Faithful Chain-of-Thought Reasoning — REFACT adaptively grounds LLM reasoning steps with context, ensuring faithful and compact outputs by integrating citations as intermediate states.
• ReTreVal: Reasoning Tree with Validation and Cross-Problem Memory for Large Language Models — Combining structured exploration, iterative refinement, explicit validation, and persistent memory significantly enhances LLM multi-step reasoning.
• ReContext: Recursive Evidence Replay as LLM Harness for Long-Context Reasoning — RECONTEXT improves LLM long-context reasoning by replaying relevant evidence without training or external memory.
• Adaptive Conformal Prediction for Improving Factuality of Generations by Large Language Models — Adaptive conformal prediction improves LLM factuality by calibrating uncertainty thresholds based on prompt characteristics, enhancing conditional coverage.
• Reliable Chain-of-Thought via Prefix Consistency — Truncating and regenerating LLM Chain-of-Thought traces reveals a reliability signal for improved answer aggregation.

See Counsel's argued verdicts on the open AI decisions leaders are weighing →

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by cs.CL updates on arXiv.org.