The World Leaks the Future: Harness Evolution for Future Prediction Agents

2026-04-17 · Source: Artificial Intelligence · Field: Technology & Digital — Artificial Intelligence & Machine Learning · Depth: Expert, quick

Summary

Milkyway is a self-evolving agent system designed to improve future prediction capabilities by updating a persistent "future prediction harness" rather than the base large language model (LLM). This system addresses the challenge of making consequential decisions before outcomes are known, a common problem in future prediction tasks. Milkyway operates by revisiting unresolved questions over time, extracting "internal feedback" from temporal contrasts between earlier and later predictions. This feedback identifies omissions in the prediction process and is used to write reusable guidance back into the harness, enabling improvements before the final outcome is known. Once a question is resolved, a "retrospective check" is performed, and the updated harness is carried forward. On the FutureX benchmark, Milkyway improved scores from 44.07 to 60.90, and on FutureWorld, scores increased from 62.22 to 77.96, achieving the best overall performance among compared methods.

Key takeaway

For research scientists developing future prediction agents, you should consider adopting a self-evolving harness approach like Milkyway. This method allows for continuous improvement of factor tracking, evidence gathering, and uncertainty handling without retraining the base LLM, significantly boosting prediction accuracy on evolving public information.

Key insights

Milkyway improves LLM future prediction by evolving a persistent harness with internal feedback, not the base model.

Principles

• Internal feedback improves predictions before outcomes are known.
• Temporal contrasts reveal prediction process omissions.

Method

Milkyway extracts internal feedback from repeated predictions on unresolved questions, writes reusable guidance to a persistent harness, and performs a retrospective check after resolution.

In practice

• Apply temporal contrasts to identify prediction gaps.
• Use a persistent harness for evolving prediction logic.

Topics

• Future Prediction Agents
• LLM Agents
• Milkyway System
• Future Prediction Harness
• Internal Feedback

Best for: Research Scientist, AI Scientist, Machine Learning Engineer

Related on AIssential

• The World Leaks the Future: Harness Evolution for Future Prediction Agents — Internal feedback from temporal prediction contrasts can improve future prediction agents before outcomes are known.
• AI Self EVOLUTION (Meta Harness) — Automating LLM harness engineering via a self-improving coding agent significantly boosts performance and efficiency.
• Continual Harness: Online Adaptation for Self-Improving Foundation Agents [R] — Self-improving agents emerge from iterative harness refinement and model-harness co-learning.
• PTCG-Bench: Can LLM Agents Master Pokémon Trading Card Game? — LLM agents show promise in complex games but struggle with sustained self-evolution and are sensitive to evaluation harness design.
• Temporal Predictive AI Agents: MILKYWAY — External harnesses enable LLMs to learn continuously and make robust future predictions without internal parameter updates.
• Adapting the Interface, Not the Model: Runtime Harness Adaptation for Deterministic LLM Agents — Adapting the LLM agent's runtime interface, not its model weights, significantly boosts performance in deterministic environments.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Artificial Intelligence.