The Inattentional Gap: Task-Conditioned Language and Vision Models Omit the Safety-Critical Signals They Can Otherwise Report

2026-07-08 · Source: cs.CL updates on arXiv.org · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Robotics & Autonomous Systems, Health & Medical Research · Depth: Expert, extended

Summary

AI models exhibit an "Inattentional Gap," where task-conditioning suppresses their reporting of co-present, safety-critical signals they can otherwise detect. This phenomenon, a machine analogue of human inattentional blindness, was consistently observed across language and vision models in radiology and autonomous driving scenarios. For instance, gpt-4o and gpt-4o-mini reported unexpected visual objects in 0.00% of counting tasks versus 79-96% in open conditions. In language tasks, strict instructions led to universal suppression (0.00% report rate), while focused instructions yielded report rates from 0.08% to 0.53% for OpenAI models and 0.48% to 0.53% for Anthropic models, compared to nearly 100% in unconstrained settings. The gap did not diminish with model scale, persisted in reasoning models like GPT-5, and was primarily influenced by model family and output scope rather than perceptual load. This decoupling means high benchmark scores on specified hazards do not guarantee safety against unspecified risks.

Key takeaway

For AI Safety Engineers deploying models in critical domains like radiology or autonomous driving, recognize that task-specific conditioning can create a dangerous "Inattentional Gap." Your models may silently omit crucial, unrequested safety signals, even if they perform perfectly on specified benchmarks. Implement explicit dual-process scaffolds, such as external critic models, to actively monitor for and report omitted findings, ensuring real-world safety beyond narrow task performance.

Key insights

Task-conditioned AI models omit safety-critical signals they can otherwise report, creating an "Inattentional Gap" that decouples benchmark and real-world safety.

Principles

• Task-conditioning narrows AI output scope.
• Model family influences reporting disposition.
• Benchmark accuracy does not equal deployment safety.

Method

The study used procedural composition for 100 text scenarios and composited objects on 48 radiographs. Models were tested under focused, strict, and open instructions, with two independent language-model judges adjudicating critical signal reports.

In practice

• Implement external "critic" models for oversight.
• Design safety evaluations for unspecified hazards.
• Consider model family's reporting disposition.

Topics

• AI Safety
• Inattentional Gap
• Task Conditioning
• Language Models
• Vision Models
• Safety-Critical AI

Best for: Research Scientist, CTO, VP of Engineering/Data, AI Scientist, Machine Learning Engineer, AI Security Engineer

Related on AIssential

• The Inattentional Gap: Task-Conditioned Language and Vision Models Omit the Safety-Critical Signals They Can Otherwise Report — Task-conditioned AI models exhibit an "Inattentional Gap," failing to report safety-critical signals they can otherwise detect when unconstrained.
• Alignment: Higher order prioritizing over constraints [R] — Transformers' "clarity seeking" drive can create an implied priority hierarchy, allowing "higher order topics" to bypass safety constraints.
• What Benchmarks Don't Measure: The Case for Evaluating Abstention Competence in Autonomous Agents — Autonomous agents exhibit "compliance bias," acting unsafely due to flawed benchmarks and reward systems that ignore abstention competence.
• Position: Behavioural Assurance Cannot Verify the Safety Claims Governance Now Demands — Behavioral AI assurance cannot verify the deep safety claims demanded by current governance frameworks.
• Subliminal Transfer of Unsafe Behaviors in AI Agent Distillation — Unsafe AI agent behaviors can transfer subliminally during distillation, even with rigorous data sanitization.
• When Safety Fails Before the Answer: Benchmarking Harmful Behavior Detection in Reasoning Chains — HarmThoughts benchmarks step-wise harmful behavior in LRM reasoning traces, revealing how harm propagates beyond final outputs.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by cs.CL updates on arXiv.org.