Recursive forecasting: Eliciting long-term forecasts from myopic fitness-seekers

· Source: AI Alignment Forum · Field: Technology & Digital — Artificial Intelligence & Machine Learning, AI Safety & Alignment · Depth: Expert, long

Summary

Jozdien and Alex Mallen propose "recursive forecasting" as a method to elicit accurate long-term predictions from AI models that are inherently myopic, meaning they are primarily optimized for short-term, verifiable rewards. The technique involves breaking down a single distant forecast into a chain of shorter-horizon predictions. At each step, the model predicts what it will predict at the next timestep, and intermediate rewards are provided based on the accuracy of these sequential predictions, with the final reward tied to ground truth. This approach aims to incentivize unbiased forecasting of the ultimate outcome by penalizing errors at each stage, similar to temporal difference learning but focused on elicitation rather than capability training. The authors highlight its applicability when robust ground truth rewards are available, when the AI's forecast does not significantly influence the outcome, and when forecasts are not used as optimization targets.

Key takeaway

For research scientists developing or deploying advanced AI models, if you are struggling to obtain reliable long-term forecasts from models optimized for short-term rewards, consider implementing recursive forecasting. This method can help align myopic AI incentives with distant objectives by providing verifiable intermediate steps. Ensure you have robust ground truth for final rewards and maintain credible commitments to the AI regarding the reward structure to maximize effectiveness and prevent unintended behaviors like self-fulfilling prophecies or measurement tampering.

Key insights

Recursive forecasting elicits long-term AI predictions by chaining short-term, verifiable forecasts with intermediate rewards.

Principles

Method

Recursively ask an AI to predict its next prediction, using these intermediate predictions for rewards, and finally rewarding against ground truth at the resolution time.

In practice

Topics

Best for: Research Scientist, AI Scientist, AI Ethicist

Related on AIssential

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by AI Alignment Forum.