Seems ICML is rejecting MANY unanimous positively rated papers [D]

2026-04-30 · Source: Machine Learning · Field: Science & Research — Artificial Intelligence & Machine Learning, Research Methodology & Innovation · Depth: Intermediate, quick

Summary

The International Conference on Machine Learning (ICML) 2024 is reportedly rejecting numerous research papers that received uniformly positive reviewer scores, including multiple instances of 4/4/4/4 ratings. This situation mirrors issues observed at previous conferences like NeurIPS. Authors and reviewers attribute this to a perceived misalignment in the review process, where the rebuttal phase incentivizes reviewers to inflate scores to avoid prolonged discussions, leading to an overall inflation of scores. Simultaneously, some initially positive reviewers are reluctant to increase scores even after concerns are addressed, resulting in papers with strong reviews still receiving low final scores. This creates a scenario where many highly-rated papers are rejected due to limited conference capacity and a complex, potentially distorted, review mechanism.

Key takeaway

For AI Scientists submitting to top-tier machine learning conferences, you should be aware that even papers with unanimous positive reviews (e.g., 4/4/4/4) face significant rejection risks due to systemic score inflation and limited capacity. Focus on crafting exceptionally clear and comprehensive rebuttals, but also prepare for the possibility of rejection despite strong reviewer feedback, and consider alternative publication venues.

Key insights

ICML 2024 is rejecting many uniformly positively rated papers due to review process distortions and capacity limits.

Principles

• Reviewer incentives can distort scoring.
• Score inflation impacts acceptance rates.

In practice

• Anticipate high rejection rates for top-tier ML conferences.
• Focus on clear, concise rebuttals to address all concerns.

Topics

• ICML Paper Rejection
• Peer Review Process
• Reviewer Score Inflation
• Rebuttal Phase Dynamics
• Conference Capacity

Best for: AI Scientist, Research Scientist, AI Student

Related on AIssential

• [D] Saw this papaer from ICLR with scores 2,2,2,4 and got accepted, HOW — Area Chair overrides in peer review can lead to controversial paper acceptance/rejection decisions, especially with low reviewer scores.
• [D] ICML paper to review is fully AI generated — AI-generated research papers challenge conference policies and reviewer assessment of technical merit and authorship.
• Intelligence Is Not the Bottleneck: Validating an LLM First-Pass Manuscript Score Against Peer-Review Outcomes — An LLM-based system can accurately predict peer-review outcomes and provide structured feedback for academic manuscripts.
• The uncritical adoption of AI in science is alarming — we urgently need guard rails — Uncritical AI adoption in science risks reduced research quality, narrowed scope, and deskilling future scientists.
• Can AI Review Improve Paper Drafting? An Empirical Study on 20 Computer Architecture Submissions — AI review can significantly align with human feedback and identify novel issues in paper drafting.
• Most Influential ICML Papers (2026-03 Version) — Citation-based ranking reveals enduring influence of ICML papers across diverse machine learning advancements.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Machine Learning.