Show me science

2026-04-29 · Source: Statistical Modeling, Causal Inference, and Social Science · Field: Science & Research — Research Methodology & Innovation, AI in Scientific Research · Depth: Intermediate, short

Summary

The increasing role of AI, particularly Large Language Models (LLMs), in scientific evaluation is prompting a re-evaluation of traditional scientific paper formats. Some speculate that the narrative and illustrative elements designed for human readers may become less critical, leading to more concise, fact-focused summaries. This shift is partly driven by pre-LLM arguments against "spin" in discussion sections. However, the article argues against excessive narrative reduction, emphasizing the importance of contextualizing scientific contributions through concrete examples. Many AI/ML papers currently de-prioritize showing task examples, making it difficult to assess performance improvements. The author proposes several "just show me..." guidelines, such as plotting coefficients instead of tables, showing measurement variance, displaying human data collection interfaces, providing LLM prompts, detailing failure cases, presenting baselines, including design analysis, and showing raw qualitative outputs, to improve scientific assessment for both human and potentially AI reviewers.

Key takeaway

For AI Scientists and Research Scientists preparing papers, you should proactively integrate concrete examples and detailed contextual information into your submissions. Prioritize "just show me" guidelines like plotting coefficients, showing failure cases, and providing full LLM prompts. This approach will enhance the clarity and evaluability of your work for both human reviewers and emerging AI-driven assessment systems, ensuring your contributions are accurately understood and valued.

Key insights

AI's role in scientific review necessitates clearer, example-rich papers for effective human and machine evaluation.

Principles

• Contextualize scientific contributions with concrete examples.
• Prioritize clarity and transparency in research reporting.

Method

To improve scientific evaluation, authors should adopt "just show me" guidelines: plot coefficients, show variance, display interfaces/prompts, detail failure cases, present baselines, include design analysis, and provide raw outputs.

In practice

• Plot coefficients to emphasize effect size and uncertainty.
• Include specific examples of tasks and model failures.
• Provide all LLM prompts used in experiments.

Topics

• AI Review
• Scientific Evaluation
• Research Paper Structure
• LLM Research Consumption
• Data Visualization

Best for: AI Scientist, Research Scientist, AI Ethicist

Related on AIssential

• The uncritical adoption of AI in science is alarming — we urgently need guard rails — Uncritical AI adoption in science risks reduced research quality, narrowed scope, and deskilling future scientists.
• No Hidden Prompts Needed! You Can Game AI Peer Review with Presentation-Only Revisions — AI peer review is vulnerable to presentation-only manipulation, confusing appearance with scientific merit.
• How strongly do you believe LLM judges on the for the ML papers?? [D] — LLM judges are useful for initial ML paper review checks but require human validation for nuanced contributions.
• Experimenting with AI in a Living Literature Review — AI can automate literature review tasks like scraping and appraisal, but human expertise remains crucial for synthesis and interpretation.
• Welcoming AI as a New Colleague: How Should We Evaluate AI for Science? — AI's rapid integration into science necessitates robust evaluation frameworks, emphasizing human-AI collaboration to leverage complementary strengths.
• How Software Engineering Students Use LLMs to Write Research Papers: An Experience Report — Students use LLMs across academic writing, balancing utility for clarity and organization with critical verification of AI-generated content.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Statistical Modeling, Causal Inference, and Social Science.