v239: "I Can't Believe It's Not Better" NeurIPS Workshop 2023

2026-06-10 · Source: Proceedings of Machine Learning Research · Field: Technology & Digital — Artificial Intelligence & Machine Learning · Depth: Expert, quick

Summary

This volume presents the proceedings from the "I Can't Believe It's Not Better: Failure Modes in the Age of Foundation Models" workshop at NeurIPS 2023, held on December 16, 2023. The papers explore diverse limitations and challenges across large language models (LLMs) and vision-language models (VLMs), including issues with ensembling LVLMs for Visual Question Answering (VQA) and the efficacy of LLMs as annotators. Discussions delve into societal impacts such as filter bubbles, affective polarization in user-personalized LLM outputs, and social bias in text-to-image foundation models. Technical concerns addressed include adversarial attacks and defenses in LLMs, the value of scaling learned optimizers, and the role of linguistic priors in compositional generalization of VLMs. Overall, the collection highlights ongoing research efforts to understand, evaluate, and mitigate the inherent shortcomings and failure modes of contemporary foundation models.

Key takeaway

The NeurIPS 2023 workshop on "Failure Modes in the Age of Foundation Models" critically examines limitations across LLMs, LVLMs, and text-to-image models. Papers detail specific challenges including filter bubbles, social bias, adversarial attacks, reasoning failures, and the cost-effectiveness of scaling learned optimizers (e.g., VeLO's 4000 TPU months). This collection provides crucial insights for AI researchers, developers, and ethicists focused on improving the robustness, fairness, and reliability of next-generation AI systems.

Topics

• Foundation Model Failure Modes
• Large Language Models
• Vision-Language Models
• Adversarial Attacks
• Social Bias

Best for: AI Scientist, Machine Learning Engineer

Related on AIssential

• Data Machina #262 — New models and frameworks are advancing LLM efficiency, reasoning, and long-form content generation.
• Identifying High-Confidence Social Biases in LLMs for Trustworthy Conversational Tutoring Agents — LLMs in tutoring agents exhibit overconfident social biases, challenging detection in conversational contexts.
• Carnegie Mellon at NeurIPS 2025 — CMU's NeurIPS 2025 contributions advance AI across diverse domains, from fundamental theory to practical applications.
• Box Maze: A Process-Control Architecture for Reliable LLM Reasoning — Architectural process-control layers can significantly improve LLM reasoning reliability and reduce adversarial failure rates.
• Pushing the Boundaries of Multiple Choice Evaluation to One Hundred Options — Scaling multiple-choice evaluations to 100 options reveals LLM competence gaps obscured by low-option benchmarks.
• Why Most Financial Chatbots Are Broken by Design — And What It Actually Takes to Fix Them — Effective conversational AI prioritizes user mental models and robust system design over raw NLP capabilities.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Proceedings of Machine Learning Research.