StructHallu-Drift: Benchmarking Structured Hallucinations Under Schema Evolution in LLMs

2026-06-25 · Source: Paper Index on ACL Anthology · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Data Science & Analytics · Depth: Expert, medium

Summary

StructHallu-Drift is a new benchmark and evaluation framework designed to study structured hallucinations in Large Language Models (LLMs) when generating outputs like JSON or SQL under schema evolution. Introduced by Mujtaba Hasan, this framework addresses the gap in semantic reliability characterization despite advances in syntactic compliance. It includes a six-category hallucination taxonomy, a controlled evaluation suite with three schema mutation severity levels, and a systematic evaluation of four LLMs ranging from 7B to 70B parameters across three structured output tasks. Experiments on 1,200 instances revealed that 39–54% of structured outputs contain semantic hallucinations. Surprisingly, schema drift severity had minimal impact on hallucination rates (~44%), while output format was a dominant factor, with SQL achieving ~85% semantic validity compared to 7–24% for schema-grounded record generation. The study also found that each model exhibits a unique hallucination fingerprint.

Key takeaway

For Machine Learning Engineers developing LLM applications that generate structured data, you must prioritize semantic reliability over mere syntactic compliance. Your current prompting setups may not adequately condition LLMs against schema drift, as hallucination rates remain high (~44%) regardless of severity. Focus on output format, as SQL generation offers ~85% semantic validity, significantly outperforming record generation (7–24%). Implement model-specific mitigation strategies, as each LLM exhibits a distinct hallucination fingerprint.

Key insights

LLMs frequently produce semantic hallucinations in structured outputs, especially with schema evolution, requiring model-specific mitigation.

Principles

• Semantic reliability is distinct from syntactic compliance.
• Schema drift severity minimally impacts hallucination rates.
• Output format significantly affects generation reliability.

Method

StructHallu-Drift evaluates LLMs by applying three severity levels of schema mutations to NL-to-structure datasets, using a six-category hallucination taxonomy to assess semantic fidelity.

In practice

• Prioritize SQL for higher semantic validity in structured generation.
• Develop model-specific hallucination mitigation strategies.
• Use the StructHallu-Drift benchmark for evaluation.

Topics

• Large Language Models
• Structured Output Generation
• Semantic Hallucinations
• Schema Drift
• SQL Generation
• Benchmarking

Best for: Research Scientist, AI Architect, AI Engineer, AI Scientist, Machine Learning Engineer, NLP Engineer

Related on AIssential

• Hallucination as Commitment Failure: Larger LLMs Misfire Despite Knowing the Answer — LLM hallucinations often result from commitment failures, where models disperse probability mass despite knowing the correct answer.
• Citation Discipline in Spec-Driven Development: A Cross-Model Empirical Study of Output Determinism and Automated Hallucination Detection in LLM-Generated Code — Mandatory inline code citations in LLM-generated code trade output determinism for automated hallucination detection.
• Accuracy, Stability, and Repeated-Run Reliability of Large Language Models on Deterministic Programming Tasks — LLM evaluation for deterministic tasks requires repeated-run stability analysis to complement conventional single-run accuracy reporting.
• LLM-as-Judge Framework for Evaluating Tone-Induced Hallucination in Vision-Language Models — Linguistic tone systematically influences VLM hallucination, with varying effects on frequency and intensity across models and tasks.
• Building Reliable Long-Form Generation via Hallucination Rejection Sampling — Rejecting hallucinated segments and resampling during generation mitigates snowballing in long-form LLM outputs.
• 5 Practical Techniques to Detect and Mitigate LLM Hallucinations Beyond Prompt Engineering — LLM hallucinations are a system problem requiring multi-layered detection and mitigation beyond prompt engineering.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Paper Index on ACL Anthology.