OmniFood-Bench: Evaluating VLMs for Nutrient Reasoning and Personalized Health Advice

2026-07-11 · Source: cs.AI updates on arXiv.org · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Robotics & Autonomous Systems, Health & Medical Research · Depth: Expert, long

Summary

OmniFood-Bench is a new benchmark designed to evaluate Large Vision-Language Models (VLMs) for nutrient reasoning and personalized health advice, addressing the "Systemic Information Asymmetry" between food's visual appearance and its nutritional composition. Constructed from the MM-Food-100K dataset, it assesses VLMs across three progressive capabilities: "Basic Perception" (Ingredients & Cooking Methods), "Quantitative Reasoning" (Portion Size & Nutritional Profiling), and "Safety-Critical Advisory" (Disease-Specific Recommendations). The benchmark evaluated six leading VLMs, including gpt-5.1, gemini-3-flash, and qwen3-vl-8B, on 1,208 samples (or 496 for closed-source models). Experiments revealed a "Semantic-Physical Gap": models achieve near-human accuracy in naming dishes but exhibit catastrophic failure in mass estimation (e.g., gpt-5.1 with 185% MAPE for raw vegetables) and frequently hallucinate benign advice for high-risk diabetic profiles, with the best model achieving only 46% accuracy in kidney disease advice. This work establishes a rigorous standard for trustworthiness in autonomous agents for public health.

Key takeaway

For AI Scientists developing VLMs for health applications, you must prioritize robust quantitative reasoning over semantic understanding. Your current models, like gpt-5.1, exhibit a "Semantic-Physical Gap" and "Safety Hallucinations" in dietary advice, making them unsafe for deployment. Focus on improving "Visual-to-Mass" estimation and integrating clinical guidelines to prevent adverse health outcomes for patients relying on your systems.

Key insights

VLMs struggle with quantitative food analysis and safety-critical health advice due to a "Semantic-Physical Gap".

Principles

• Visual appearance alone is insufficient for nutritional composition.
• Safety alignment is a fundamental requirement for healthcare AI.
• "Sycophantic" model behavior can lead to safety hallucinations.

Method

OmniFood-Bench evaluates VLMs hierarchically: "Basic Perception" (ingredients, cooking methods), "Quantitative Estimation" (portion size, nutritional profile via MAPE), and "Advanced Advisory" (disease-specific recommendations via classification accuracy).

In practice

• Focus on "Visual-to-Mass" estimation for VLM improvement.
• Integrate structured nutritional knowledge bases.
• Employ Retrieval-Augmented Generation (RAG) for advice.

Topics

• Large Vision-Language Models
• Nutrient Reasoning
• Personalized Health Advice
• OmniFood-Bench
• AI Safety Alignment
• Quantitative Estimation

Best for: CTO, VP of Engineering/Data, Director of AI/ML, AI Scientist, Research Scientist, AI Ethicist

Related on AIssential

• OmniFood-Bench: Evaluating VLMs for Nutrient Reasoning and Personalized Health Advice — VLMs exhibit a "Semantic-Physical Gap," failing catastrophically in food mass estimation and safety-critical health advice despite visual recognition accuracy.
• Thinking Like a Botanist: Challenging Multimodal Language Models with Intent-Driven Chain-of-Inquiry — Structured, intent-driven inquiry significantly enhances MLLM diagnostic accuracy and reasoning efficiency in botanical pathology.
• RuleSafe-VL: Evaluating Rule-Conditioned Decision Reasoning in Vision-Language Content Moderation — Content moderation evaluation needs to shift from final-label scoring to diagnostic assessment of rule-conditioned decision reasoning.
• Beyond Accuracy: Evaluating Visual Grounding In Multimodal Medical Reasoning — Accuracy-only RLVR in medical VQA degrades visual grounding despite improving benchmark accuracy.
• VG-CoT: Towards Trustworthy Visual Reasoning via Grounded Chain-of-Thought — VG-CoT enhances LVLM trustworthiness by grounding multi-step reasoning in explicit visual evidence through an automated pipeline.
• MM-CondChain: A Programmatically Verified Benchmark for Visually Grounded Deep Compositional Reasoning — Deep compositional reasoning in MLLMs remains a significant challenge, especially with increasing complexity.

See Counsel's argued verdicts on the open AI decisions leaders are weighing →

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by cs.AI updates on arXiv.org.