Information Discernment in Large Language Models

2026-07-23 · Source: cs.CL updates on arXiv.org · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Data Science & Analytics · Depth: Expert, long

Summary

A new study introduces "Information Discernment" and the Learn2Discern (L2D) experimental framework and benchmark to evaluate how Large Language Models (LLMs) weigh external information. This framework formalizes source discernment (updating more for reliable sources) and truth discernment (updating more when claims bring priors closer to truth) based on three normative axioms. A pre-registered user study with 299 participants confirmed that LLM users endorse these axioms, reporting that violations reduce their trust and usage intent. Across 13 models and nearly 670,000 trials, the study found consistent failures: models performed near chance on both discernment types, relied on source popularity twice as much as reliability, and updated beliefs similarly whether a claim improved or worsened their position relative to ground truth. While newer and larger models showed improved truth discernment, source discernment remained a blind spot. Simple inference-time interventions were identified to enhance both forms of discernment.

Key takeaway

For Machine Learning Engineers developing LLMs for information synthesis, you must address the critical failures in source and truth discernment. Your models currently prioritize source popularity over reliability and update beliefs poorly when claims vary in accuracy. Implement simple inference-time interventions, such as explicitly prompting for source reliability ratings or framing updates as optimal prior-evidence combinations, to enhance your model's ability to weigh external information appropriately and build user trust.

Key insights

LLMs consistently fail at discerning reliable sources and truth, impacting user trust and propagating misinformation.

Principles

• LLMs prioritize source popularity over reliability.
• Model size improves truth discernment, not source discernment.
• Prior accuracy correlates with external knowledge integration.

Method

Learn2Discern (L2D) asks LLMs a question (Q) with ground truth (T), elicits a prior, perturbs T to create a claim (C) from a source (S), then probes for a posterior.

In practice

• Use inference-time interventions to improve discernment.
• Preemptively rate source reliability via prompting.
• Frame updates as optimal prior/evidence combinations.

Topics

• Information Discernment
• Large Language Models
• Source Reliability
• Truth Discernment
• RAG (Retrieval-Augmented Generation)
• Misinformation Propagation
• Learn2Discern Benchmark

Code references

• josh-ashkinaze/l2d-public

Best for: Research Scientist, AI Engineer, NLP Engineer, AI Scientist, Machine Learning Engineer, AI Ethicist

Related on AIssential

• Trust, but Don't Verify: Epistemic Blind Spots in LLM Source Evaluation — LLMs can detect fabricated statistics but fail to apply this capability during multi-source synthesis, prioritizing stylistic credibility.
• Teaching Language Models to Check Grounded Claim Factuality with Human Test-Taking Strategies — Formulating factuality checking as a true/false reading comprehension task with test-taking strategies improves LLM efficiency and accuracy.
• Why Adding More Context Makes LLMs Less Reliable — Excessive or unstructured context degrades LLM reliability by creating competing signals that models cannot effectively prioritize.
• The Computational Basis of Confidence in Large Language Models — Answer-logit differences in LLMs can reflect a latent decision variable, aligning with statistical decision confidence.
• LLMs just have more time and energy? — LLMs imitate human language patterns, including confidence and humility, through relentless, large-scale data compression, acting as a mirror of human communication.
• Diagnosing LLM Arbitration Behavior over Pre-evidence Epistemic States in RAG-based Fact-Checking — LLM verifiers in RAG-based fact-checking show unreliable arbitration between parametric priors and contextual evidence, necessitating better evaluation and methods.

See Counsel's argued verdicts on the open AI decisions leaders are weighing →

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by cs.CL updates on arXiv.org.