It's not the Language Model, it's the Tool: Deterministic Mediation for Scientific Workflows

2026-05-15 · Source: cs.AI updates on arXiv.org · Field: Science & Research — Research Methodology & Innovation, Engineering & Applied Sciences, Artificial Intelligence & Machine Learning · Depth: Expert, extended

Summary

A new paper from Marios Adamidis, Danae Katrisioti, Yannis Tzitzikas, and Emmanuel Stratakis, published May 13, 2026, introduces "typed mediation," a pattern for integrating language models (LLMs) into scientific workflows to ensure reproducibility and data privacy. The approach involves LLMs orchestrating deterministic, researcher-defined tools rather than generating analytical code. These tools, developed through structured interviews, encode exact scientific procedures for specific instruments and run on local infrastructure, addressing issues with proprietary software licenses and data sensitivity. The FORTHought platform, implementing this pattern, was evaluated against commercial LLMs (GPT-5.5, Claude Sonnet 4.6, Gemini 3.1 Pro) for photoluminescence analysis. FORTHought consistently produced identical results across four runs (σb=0), while commercial platforms exhibited significant variance in numerical output and analytical methodology, or failed entirely. This method reduced analysis time from weeks to minutes for a photoluminescence workflow and is also deployed for scanning electron microscopy.

Key takeaway

For AI Engineers and Research Scientists developing LLM-assisted scientific tools, you should adopt a typed mediation architecture. This approach ensures analytical reproducibility by having the LLM orchestrate deterministic, researcher-defined tools on local infrastructure, rather than generating variable code. This guarantees consistent results, addresses data privacy, and significantly reduces analysis time, making LLMs viable for critical experimental workflows where precision is non-negotiable.

Key insights

Typed mediation uses LLMs to orchestrate deterministic, researcher-defined tools for reproducible scientific analysis, ensuring consistent results.

Principles

• Reproducibility is paramount in scientific analysis.
• LLM stochasticity prevents consistent code generation.
• Deployment topology is critical for scientific tools.

Method

Encode researcher's exact workflow into a typed tool via structured interviews. LLM orchestrates tool calls through a schema, executing deterministic analysis on local infrastructure. This separates stochastic reasoning from deterministic computation.

In practice

• Port existing manual workflows into typed tools.
• Use Model Context Protocol (MCP) for tool interfaces.
• Tailor skill files to constrain LLM behavior.

Topics

• Typed Mediation Pattern
• Scientific Reproducibility
• LLM Tool Orchestration
• Scientific Data Analysis
• Photoluminescence Spectroscopy

Best for: Research Scientist, AI Scientist, AI Engineer

Related on AIssential

• It's not the Language Model, it's the Tool: Deterministic Mediation for Scientific Workflows — Deterministic tools orchestrated by language models ensure reproducibility in scientific workflows, unlike direct code generation.
• From Data to Theory: Autonomous Large Language Model Agents for Materials Science — Autonomous LLM agents can perform end-to-end scientific theory development, from equation selection to code execution and validation.
• Ontology-constrained multi-LLM scoring of hypothesis support in the predictive processing literature — A multi-LLM council with an expert-guided ontology can quantitatively map scientific literature's support for hypotheses, revealing structured agreement and disagreement.
• Automated reproducibility assessments in the social and behavioral sciences using large language models — Large language models can automate reproducibility assessments in social sciences, outperforming human reanalysts in qualitative agreement.
• Recognizing reproducibility and reusability in times of fast science — Reproducibility and reviewability are paramount for scientific integrity, especially with LLM integration.
• Towards Generalist Agents for Accelerating Scientific Discovery171 — LLMs can accelerate scientific discovery by generating and refining hypotheses within agentic, iterative workflows.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by cs.AI updates on arXiv.org.