Would a 2000-2021 ML paper even get accepted today? [D]

2026-05-14 · Source: Machine Learning · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Research Methodology & Innovation · Depth: Expert, medium

Summary

A discussion among machine learning professionals explores whether ML papers accepted between 2000 and 2021 would meet today's publication standards. Many contributors suggest the bar has significantly risen due to increased competition, higher expectations for evaluation, and more rigorous baselines. However, some argue that the bar for Large Language Model (LLM) papers, particularly those focused on prompting or "harness engineering," has lowered, blurring the line between academic research and engineering white papers. The conversation also touches on the increased financial cost of research, the perceived decline in reviewer quality, and the challenge of publishing simple yet novel methods that might be deemed "too incremental" despite high potential impact, such as AdamW.

Key takeaway

For machine learning researchers preparing submissions, recognize that current peer review demands extensive compute and exhaustive ablation studies, often favoring large-scale engineering projects over single clever ideas. Your work must demonstrate significant empirical rigor and comprehensive evaluation, even if it means substantial resource investment, to navigate the increasingly competitive publication landscape and avoid rejection for perceived "incrementalism."

Key insights

ML paper acceptance standards have shifted, with higher expectations for traditional research but a potentially lower bar for LLM-centric engineering.

Principles

• Research costs have substantially increased.
• Reviewer quality is a critical factor.
• Incremental novelty can be overlooked.

In practice

• Focus on comprehensive evaluation and strong baselines.
• Distinguish academic papers from engineering white papers.
• Consider the financial cost of compute for experiments.

Topics

• ML Publication Standards
• Paper Acceptance Criteria
• LLM Research
• Peer Review Quality
• Computational Costs

Best for: AI Scientist, Machine Learning Engineer, Research Scientist

Related on AIssential

• Seems ICML is rejecting MANY unanimous positively rated papers [D] — ICML 2024 is rejecting many uniformly positively rated papers due to review process distortions and capacity limits.
• Are modern ML PhDs becoming too incremental, or is this just what research looks like now? [D] — ML PhDs often prioritize incremental benchmark improvements due to academic and industry incentives.
• Intelligence Is Not the Bottleneck: Validating an LLM First-Pass Manuscript Score Against Peer-Review Outcomes — An LLM-based system can accurately predict peer-review outcomes and provide structured feedback for academic manuscripts.
• The uncritical adoption of AI in science is alarming — we urgently need guard rails — Uncritical AI adoption in science risks reduced research quality, narrowed scope, and deskilling future scientists.
• The State of Peer Review in Empirical Software Engineering: A Community Survey on Review Load, Quality, and GenAI Use — The ESE peer review system faces unsustainable workload, quality issues, and ethical dilemmas exacerbated by GenAI, demanding urgent systemic changes.
• Can AI Review Improve Paper Drafting? An Empirical Study on 20 Computer Architecture Submissions — AI review can significantly align with human feedback and identify novel issues in paper drafting.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Machine Learning.