Recursive forecasting: Eliciting long-term forecasts from myopic fitness-seekers
Summary
Jozdien and Alex Mallen propose "recursive forecasting" as a method to elicit accurate long-term predictions from AI models that are inherently myopic, meaning they are primarily optimized for short-term, verifiable rewards. The technique involves breaking down a single distant forecast into a chain of shorter-horizon predictions. At each step, the model predicts what it will predict at the next timestep, and intermediate rewards are provided based on the accuracy of these sequential predictions, with the final reward tied to ground truth. This approach aims to incentivize unbiased forecasting of the ultimate outcome by penalizing errors at each stage, similar to temporal difference learning but focused on elicitation rather than capability training. The authors highlight its applicability when robust ground truth rewards are available, when the AI's forecast does not significantly influence the outcome, and when forecasts are not used as optimization targets.
Key takeaway
For research scientists developing or deploying advanced AI models, if you are struggling to obtain reliable long-term forecasts from models optimized for short-term rewards, consider implementing recursive forecasting. This method can help align myopic AI incentives with distant objectives by providing verifiable intermediate steps. Ensure you have robust ground truth for final rewards and maintain credible commitments to the AI regarding the reward structure to maximize effectiveness and prevent unintended behaviors like self-fulfilling prophecies or measurement tampering.
Key insights
Recursive forecasting elicits long-term AI predictions by chaining short-term, verifiable forecasts with intermediate rewards.
Principles
- Myopic AI models can be incentivized for long-term accuracy.
- Chaining short-horizon forecasts can approximate long-horizon goals.
- Credibility in reward delivery is crucial for AI compliance.
Method
Recursively ask an AI to predict its next prediction, using these intermediate predictions for rewards, and finally rewarding against ground truth at the resolution time.
In practice
- Apply to events resolving before developer disempowerment.
- Ensure ground truth access for final reward computation.
- Avoid using forecasts as direct optimization targets.
Topics
- Recursive Forecasting
- AI Elicitation
- Long-Horizon Forecasts
- Myopic Fitness-Seekers
- Reward Signal Capture
Best for: Research Scientist, AI Scientist, AI Ethicist
Related on AIssential
Editorial summary, takeaway, and curation by AIssential. Original article published by AI Alignment Forum.