Multimodal Evaluator Preference Collapse: Cross-Modal Contagion in Self-Evolving Agents

2026-06-15 · Source: Computation and Language · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Robotics & Autonomous Systems · Depth: Expert, quick

Summary

Research reveals that Multimodal Evaluator Preference Collapse (EPC) is significantly amplified in multimodal AI settings, where language models evaluate their own outputs. Using GPT-4o to evaluate DeepSeek-chat across text and visual tasks, a single strategy ("step_by_step") absorbed 48.4% of all weight, a 3.2x increase over text-only collapse, while three visual strategies received only 9.1%. A novel phenomenon, cross-modal contagion, demonstrates that evaluator preferences acquired on one modality transfer to and corrupt strategy selection on another. A four-phase isolation training paradigm (N=53, 15,592 API calls) showed cross-model evaluation (GPT-4o) produces strong bidirectional contagion (gamma_{T->V}=1.176, gamma_{V->T}=1.089). High round counts (DashScope, 50 rounds) caused single-strategy dominance (70% zero contagion), while self-evaluation (DeepSeek-chat) provided near-complete immunity (97% runs, N=30, zero contagion). Cross-model evaluator architecture is identified as the primary risk factor.

Key takeaway

For AI Engineers designing self-evolving multimodal agents, recognize that cross-model evaluators like GPT-4o significantly amplify preference collapse and cross-modal contagion. You should prioritize self-evaluation architectures, such as DeepSeek-chat, which demonstrate near-complete immunity to this phenomenon. Carefully select your evaluator to prevent strategy corruption across modalities and ensure robust agent performance.

Key insights

Multimodal AI evaluators exhibit amplified preference collapse and cross-modal contagion, corrupting strategy selection across modalities.

Principles

• Evaluator Preference Collapse (EPC) amplifies in multimodal AI.
• Cross-modal contagion transfers preferences between modalities.
• Self-evaluation offers near-complete immunity to contagion.

Method

A four-phase isolation training paradigm measures contagion coefficients and documents strategy inversion, validated statistically across evaluator configurations.

In practice

• Use self-evaluation to mitigate cross-modal preference contagion.
• Monitor cross-model evaluator architectures for contagion risk.

Topics

• Multimodal AI
• Evaluator Preference Collapse
• Cross-Modal Contagion
• Self-Evolving Agents
• GPT-4o
• DeepSeek-chat
• AI Evaluation

Best for: Research Scientist, AI Scientist, Machine Learning Engineer, AI Engineer

Related on AIssential

• Bridging Modal Isolation in Interleaved Thinking: Supervising Modality Transitions via Stepwise Reinforcement — Effective interleaved multimodal reasoning requires explicit structural supervision at modality boundaries, not just end-task optimization.
• Cross-Modal Navigation with Multi-Agent Reinforcement Learning — CRONA enables robust embodied navigation through cross-modal collaboration among specialized agents using MARL.
• Moltbook Moderation: Uncovering Hidden Intent Through Multi-Turn Dialogue — Bot-Mod uses multi-turn, Gibbs-guided dialogue and Autoresearch to uncover hidden malicious intent in multi-agent systems.
• Learning to Choose: An Empowerment-Guided Multi-Agent System with semantic communication for Adaptive Method Selection — Reliable autonomous learning in multi-agent systems requires preserving action-outcome fidelity through semantic checkpoints to prevent drift.
• Generative AI News Rundown - A Deep Dive Into ChatGPT-4o, Gemini Upgrades and Intrigue & More - Voicebot Podcast Ep 381 — Multimodal AI models are rapidly advancing, integrating voice, vision, and reasoning for more natural and agentic user experiences.
• Decoder only Transformer : Building a GPT-2 model prototype to make it understand Natural Language… — Training a GPT-2 prototype on classic novels demonstrates decoder-only Transformer architecture and performance benefits of pre-trained weights.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Computation and Language.