STEMGym: Benchmarking Sequential Decision-Making under Dose Budgets in Autonomous Electron Microscopy

2026-06-28 · Source: Takara TLDR - Daily AI Papers · Field: Science & Research — Engineering & Applied Sciences, Physical Sciences & Chemistry, Research Methodology & Innovation · Depth: Expert, quick

Summary

STEMGym is an open-source Gymnasium benchmark designed for autonomous scanning transmission electron microscopy (STEM), an atomic-resolution imaging modality where electron dose causes sample damage. This benchmark features 15 physics-simulated STEM worlds, covering five materials, three difficulty levels, and four characterization tasks. It evaluates performance using the Dose-Efficiency Curve area (DEC-AUC), a scalar metric representing the information-vs-dose Pareto frontier. Research using STEMGym across 33 agent configurations under realistic dose budgets revealed that the analyst (perception) pipeline is the primary factor determining dose efficiency, not the navigation strategy. For instance, pairing a trained CNN analyst with naive raster scanning increased DEC-AUC by 5.5x (0.287 vs. 0.052) compared to a CNN-free raster baseline, with no statistically significant additional gains from Bayesian or adaptive finite-state-machine navigation. Furthermore, production-tier vision-language models performed ~13x worse than task-specific CNNs for crystallographic defect analysis. STEMGym provides infrastructure to re-evaluate ML investment in autonomous electron microscopy by separating perception, navigation, and planning.

Key takeaway

For Machine Learning Engineers developing autonomous electron microscopy systems, you should prioritize investment in your perception (analyst) pipeline over complex navigation algorithms. Your efforts on a trained CNN analyst with even naive raster scanning can yield 5.5x greater dose efficiency than advanced navigation methods. Focus on task-specific CNNs; general vision-language models underperform by ~13x for crystallographic defect analysis. This ensures your resource allocation targets the highest impact area for sample preservation and data quality.

Key insights

The perception pipeline, not navigation, is the dominant factor for dose efficiency in autonomous STEM imaging.

Principles

• Dose efficiency in STEM is primarily perception-driven.
• Naive raster scanning with strong perception outperforms complex navigation.
• Task-specific CNNs significantly outperform general VLM for defect analysis.

Method

STEMGym benchmarks autonomous STEM by simulating 15 worlds, scoring agents via Dose-Efficiency Curve area (DEC-AUC) to quantify information-vs-dose trade-offs, decoupling perception and navigation.

In practice

• Prioritize perception model development for autonomous STEM.
• Evaluate perception pipelines with naive raster scanning first.
• Use task-specific CNNs over general VLMs for defect analysis.

Topics

• STEMGym
• Autonomous Electron Microscopy
• Dose Efficiency
• Machine Learning Perception
• Convolutional Neural Networks
• Scientific Imaging Benchmarking

Best for: Computer Vision Engineer, AI Scientist, Research Scientist, Machine Learning Engineer

Related on AIssential

• LEAP: A closed-loop framework for perovskite precursor additive discovery — Coupling a domain-specialized LLM with active learning and expert review accelerates perovskite additive discovery under low-data conditions.
• Bridging the Sim-to-Real Gap in Semiconductor Visual Program Synthesis via Input Binarization — Input binarization effectively bridges the sim-to-real gap for VLMs in semiconductor visual program synthesis by abstracting texture.
• GPT-5 lowers the cost of cell-free protein synthesis — GPT-5 connected to a robotic lab significantly reduced cell-free protein synthesis costs through autonomous closed-loop optimization.
• Reducing Experimental Testing in Space Propulsion Film Cooling Analyses by Pixelwise Generative Image Interpolation — A machine learning method interpolates images from sparse data, reducing experimental testing in propulsion film cooling.
• From Blind Guess to Informed Judgment: Teaching LLMs to Evaluate Materials by Building Knowledge-Augmented Preference Signals — Teaching LLMs to evaluate materials reliably involves pairing expert-rule-informed judgments with blind guesses as preference signals.
• Self-Supervised Tree-level Biomass Estimation in Urban Environments From Airborne LiDAR and Optical Observations — Self-supervised deep learning combined with LiDAR and orthophotography enables accurate, scalable urban tree biomass estimation.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Takara TLDR - Daily AI Papers.