[D] ICML 2026 Review Discussion

2026-03-24 · Source: Machine Learning · Field: Technology & Digital — Artificial Intelligence & Machine Learning · Depth: Advanced, short

Summary

ICML 2026 paper reviews are being released today, March 24th AoE, prompting discussions among researchers about their experiences. A significant point of contention is the perceived noisiness and variability within the peer review system, particularly in machine learning. One author reported a paper receiving a lowest score due to a reviewer's mistaken belief that the authors injected hidden text, which was actually a conference-added watermark designed to detect LLM-generated reviews. Researchers are noting a mix of relief and frustration, with some observing that reviewer quality variance can exceed paper quality variance, leading to inconsistent outcomes. The high volume of submissions and the broad scope of ML subfields are cited as factors contributing to these challenges, with some suggesting potential solutions like AI-assisted pre-filtering or more powerful area chairs. Scores are being reported across a range, with some papers receiving strong acceptances and others facing rejections despite extensive work.

Key takeaway

For research scientists navigating the ICML 2026 review process, prioritize actionable feedback over overall scores, as review quality can be highly variable. Be aware that "hidden text" concerns might stem from conference-added watermarks for LLM detection, not author misconduct. If your paper receives inconsistent feedback across conferences, it highlights the randomness of the system, so focus on refining your work based on constructive criticism rather than being discouraged by subjective scores.

Key insights

ML peer review suffers from high variance in reviewer quality and increasing submission volumes.

Principles

• Review systems are inherently noisy.
• Actionable feedback is more valuable than scores.

In practice

• Focus on actionable suggestions in reviews.
• Recognize conference watermarking for LLM detection.

Topics

• ICML 2026
• Peer Review
• LLM Policy
• Reviewer Bias
• Conference Submissions

Best for: Research Scientist, AI Researcher, AI Scientist, AI Student

Related on AIssential

• [D] Has anyone received their ICML papers to review yet? — ICML 2026 is using hidden prompt injections to detect LLM-assisted paper reviews, raising ethical concerns.
• [D] ICML 2026: Policy A vs Policy B impact on scores discussion — LLM-assisted peer review may lead to more lenient scores and polished reviews compared to stricter human-only policies.
• ICLR 2026 Response to Security Incident — An OpenReview API exploit led to a major ICLR security breach, compromising anonymity and academic integrity.
• Impact of large language models on peer review opinions from a fine-grained perspective: Evidence from top conference proceedings in AI — LLMs are making peer reviews longer and more fluent but reducing focus on deep evaluative dimensions.
• Intelligence Is Not the Bottleneck: Validating an LLM First-Pass Manuscript Score Against Peer-Review Outcomes — An LLM-based system can accurately predict peer-review outcomes and provide structured feedback for academic manuscripts.
• Major conference catches illicit AI use — and rejects hundreds of papers — Watermarking can detect AI-generated content in peer review, leading to policy enforcement and paper rejections.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Machine Learning.