Disparities In Negation Understanding Across Languages In Vision-Language Models

2026-04-21 · Source: Computation and Language · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Computational Linguistics · Depth: Expert, quick

Summary

Vision-language models (VLMs) frequently exhibit an "affirmation bias," favoring positive captions over negative ones, a phenomenon previously studied primarily in English. A new human-verified multilingual negation benchmark has been introduced, covering seven typologically diverse languages: English, Mandarin Chinese, Arabic, Greek, Russian, Tagalog, and Spanish. This benchmark was used to evaluate three VLMs: CLIP, SigLIP, and MultiCLIP. Results indicate that standard CLIP performs at or below chance for non-Latin-script languages, whereas MultiCLIP demonstrates the highest and most consistent accuracy across all languages. Additionally, the SpaceVLM negation correction method was assessed, showing significant improvements for English, Greek, Spanish, and Tagalog, but varied effectiveness in other languages. This variability highlights how linguistic features like morphology, script, and negation structure influence model performance and the efficacy of corrective measures, underscoring the need for multilingual benchmarks as VLMs are deployed globally.

Key takeaway

For research scientists developing or deploying vision-language models globally, you should prioritize comprehensive multilingual evaluation beyond English. Your models' negation understanding will vary significantly across languages due to morphological and script differences, impacting fairness. Use benchmarks like the one presented to ensure solutions like SpaceVLM are effective for your target linguistic communities, rather than assuming universal applicability.

Key insights

VLMs show affirmation bias, with negation understanding varying significantly across diverse languages and models.

Principles

• Affirmation bias is a common VLM failure mode.
• Linguistic properties impact VLM negation understanding.
• Multilingual benchmarks are crucial for global VLM deployment.

Method

A human-verified multilingual negation benchmark was created across seven languages to evaluate CLIP, SigLIP, MultiCLIP, and SpaceVLM for negation understanding.

In practice

• Prioritize MultiCLIP for robust multilingual negation.
• SpaceVLM improves negation for specific languages.
• Test VLM solutions across diverse linguistic structures.

Topics

• Vision-Language Models
• Negation Understanding
• Multilingual Benchmarks
• Affirmation Bias
• Cross-Lingual Disparities

Best for: Computer Vision Engineer, Research Scientist, AI Scientist, NLP Engineer, AI Ethicist

Related on AIssential

• Disparities In Negation Understanding Across Languages In Vision-Language Models — VLMs exhibit affirmation bias, with negation understanding varying significantly across typologically diverse languages.
• Not Truly Multilingual: Script Consistency as a Missing Dimension in VLM Evaluation — VLMs exhibit substantial script-dependent bias, failing to consistently process identical content across different orthographies.
• Why Far Looks Up: Probing Spatial Representation in Vision-Language Models — Vision-language models consistently conflate vertical image position with distance, hindering robust spatial reasoning.
• Diagnosing Visual Ignorance in Vision-Language Models — VLMs' visual ignorance results from internal routing failures and benchmarks that reward language-prior reliance.
• Scale Can't Overcome Pragmatics: The Impact of Reporting Bias on Vision-Language Reasoning — Reporting bias in VLM training data hinders reasoning, which scaling alone cannot resolve.
• A Comparison of Methods to Bias Translation Toward Portuguese Variants — Biasing MT towards minority language variants is achievable through various lifecycle interventions.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Computation and Language.