It's not the Language Model, it's the Tool: Deterministic Mediation for Scientific Workflows

2026-05-13 · Source: Artificial Intelligence · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Data Science & Analytics · Depth: Advanced, quick

Summary

A new pattern called "typed mediation" is proposed to address the lack of reproducibility in scientific analyses generated by language models. This approach involves language models orchestrating deterministic tools, rather than directly generating analytical code, to ensure consistent results. The tools encapsulate a researcher's exact procedure for a specific instrument, derived through structured interviews. Evaluation across four platforms, including three commercial foundation models, demonstrated that the typed tool produced identical photoluminescence analysis results across multiple runs with the same prompt. In contrast, commercial platforms exhibited variations in numerical output and analytical methodology, or failed to produce valid results. This pattern has been successfully deployed for approximately six months on two instruments handling challenging proprietary binary formats and licensed software, reducing analysis time from weeks to minutes while guaranteeing identical outputs.

Key takeaway

For AI Architects and Machine Learning Engineers building scientific applications where reproducibility is paramount, adopting the typed mediation pattern is crucial. This approach ensures consistent analytical results by having language models orchestrate deterministic tools, rather than generating variable code. You should consider this pattern, especially when dealing with proprietary data formats or licensed software that necessitates local infrastructure deployment, to achieve reliable, repeatable scientific outcomes.

Key insights

Deterministic tools orchestrated by language models ensure reproducibility in scientific workflows, unlike direct code generation.

Principles

• Reproducibility is mandatory in scientific workflows.
• Deployment topology is a structural requirement.
• Tools encode exact researcher procedures.

Method

The language model selects and parameters a deterministic tool. The tool, encoding a researcher's procedure for an instrument, then produces the result, ensuring identical outputs upon regeneration.

In practice

• Use typed mediation for reproducible scientific analysis.
• Integrate proprietary formats with local infrastructure.
• Reduce analysis time from weeks to minutes.

Topics

• Typed Mediation
• Scientific Workflows
• Language Model Reproducibility
• Deterministic Tools
• Photoluminescence Analysis

Best for: AI Architect, Machine Learning Engineer, AI Scientist, Research Scientist, AI Engineer

Related on AIssential

• It's not the Language Model, it's the Tool: Deterministic Mediation for Scientific Workflows — Typed mediation uses LLMs to orchestrate deterministic, researcher-defined tools for reproducible scientific analysis, ensuring consistent results.
• COMPOSE: Composing Future Theorems from Citations and Formal Structure — Combining scientific citation graphs with formal theorem structures significantly improves future mathematical claim generation.
• The embrace of open science: An analysis of a decade of AI research and 56 800 conference papers — AI research documentation practices improved significantly from 2014-2024, driven by a broader open science movement.
• Externalizing Research Synthesis and Validation in AI Scientists through a Research Harness — AI scientific workflows require externalized, inspectable synthesis and validation for accountability.
• Radically Better Reasoning: Elicit's Andreas Stuhlmüller & Jungwon Byun on World Models for Research — Elicit ensures trusted AI reasoning in scientific research via process supervision, domain-specific primitives, and inspectable world models.
• Recognizing reproducibility and reusability in times of fast science — Reproducibility and reviewability are paramount for scientific integrity, especially with LLM integration.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Artificial Intelligence.