Instructions for (ICML) workshop reviews [D]

2026-05-20 · Source: Machine Learning · Field: Science & Research — Research Methodology & Innovation · Depth: Advanced, quick

Summary

An ICML workshop reviewer, Ok-Painter573, initiated a discussion seeking clarification on review guidelines, noting the absence of specific criteria, grading scales, or structural instructions for their assigned workshop. The inquiry explored whether standard ICML conference reviewer instructions, typically more rigorous, should be applied. Responses from other reviewers indicated that workshop review processes vary significantly, not only from main conferences but also among different workshops. Some workshops prioritize paper relevance and interest, with less emphasis on novelty or rigor, while others focus on readability for competition results. Conversely, larger workshops might demand conference-level scrutiny, including novelty and technical soundness. The consensus emphasized the necessity of directly consulting the workshop chairs for definitive guidance tailored to their specific event.

Key takeaway

For AI Scientists or Research Scientists reviewing for an ICML workshop, you must proactively contact the workshop chairs for specific review guidelines. Do not assume standard conference criteria apply, as workshops vary greatly in their emphasis on novelty, rigor, or relevance. Clarifying expectations upfront ensures your review aligns with the workshop's unique objectives, preventing misaligned feedback and potential delays.

Key insights

ICML workshop review standards vary widely; always confirm expectations with chairs.

Principles

• Workshop review criteria are highly variable.
• Relevance and interest can outweigh novelty.
• Competition papers prioritize readability.

Method

The recommended procedure for reviewers is to directly contact the workshop chairs to ascertain specific review criteria, grading scales, and expectations, as these vary widely.

In practice

• Consult workshop chairs for guidelines.
• Check OpenReview page for instructions.
• Adapt review focus to workshop type.

Topics

• ICML Workshops
• Peer Review Guidelines
• Academic Conferences
• Reviewer Instructions
• OpenReview Platform
• Research Evaluation

Best for: AI Scientist, Research Scientist

Related on AIssential

• ICML 2026 Decision [D] — The ICML 2026 decision period is marked by high anxiety and frustration over the peer review process.
• [D] ICML paper to review is fully AI generated — AI-generated research papers challenge conference policies and reviewer assessment of technical merit and authorship.
• Please follow the rules for ARR/ACL papers — Adherence to publication guidelines is crucial for research quality and reviewer efficiency.
• Can AI Review Improve Paper Drafting? An Empirical Study on 20 Computer Architecture Submissions — AI review can significantly align with human feedback and identify novel issues in paper drafting.
• Workshops at ICLR 2026 — ICLR 2026 accepted 40 diverse workshops from 135 valid submissions, reflecting significant growth and rigorous review.
• On the Limits of LLM-as-Judge for Scientific Novelty Assessment — LLMs are unreliable judges of scientific novelty for research questions, often creating a "novelty mirage."

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Machine Learning.