Plan Before You Trade: Inference-Time Optimization for RL Trading Agents

2026-05-14 · Source: stat.ML updates on arXiv.org · Field: Finance & Economics — Capital Markets & Investment Management, FinTech & Digital Financial Services · Depth: Expert, quick

Summary

A new framework called FPILOT (Financial Plugin Inference-time Learning for Optimal Trading) has been proposed to enhance reinforcement learning (RL) agents for portfolio management. Submitted on May 12, 2026, FPILOT addresses the limitation of static RL policies by integrating inference-time optimization using price forecasts. Inspired by Model Predictive Control (MPC), it leverages a predictive model to generate multi-step price trajectories, assuming an agent's portfolio allocation minimally impacts future prices. At each decision step, FPILOT constructs an allocation-based imagined return objective and optimizes the policy before executing a trade, without requiring agent retraining. Evaluated on the TradeMaster DJ30 benchmark across five policy learning algorithms, FPILOT consistently improved total return and risk-adjusted metrics like Sharpe, Sortino, and Calmar ratios, with stochastic policies showing greater benefits. Performance gains also correlated directly with the quality of synthetic forecasts.

Key takeaway

For quantitative analysts and AI scientists developing portfolio management systems, FPILOT offers a significant enhancement to existing RL trading agents. You should consider integrating this inference-time optimization framework to adapt your pre-trained policies to real-time price forecasts, potentially boosting total returns and improving risk-adjusted metrics like Sharpe and Sortino ratios without requiring costly retraining. Focus on improving your financial forecasting models, as higher forecast quality directly translates to better trading performance.

Key insights

FPILOT enhances RL trading agents by integrating inference-time optimization with price forecasts, improving returns and risk metrics.

Principles

• Future prices are largely independent of one agent's actions.
• Inference-time optimization can adapt pre-trained policies.
• Forecaster quality directly correlates with trading performance gains.

Method

FPILOT uses a predictive model to generate multi-step price trajectories, then optimizes the pre-trained policy at inference-time based on an imagined return objective before executing a single trade step.

In practice

• Apply FPILOT to existing RL trading agents.
• Prioritize high-quality financial forecasting models.
• Consider FPILOT for stochastic policy improvements.

Topics

• FPILOT
• Reinforcement Learning
• Portfolio Management
• Inference-time Optimization
• Model Predictive Control

Best for: AI Scientist, Data Scientist, Research Scientist

Related on AIssential

• QPILOTS: Efficient Test-Time Q-Steering for Flow Policies — QPILOTS efficiently steers flow-matching policies at inference time by computing critic gradients on projected clean actions.
• Addressing Market Regime Changes and Heavy-Tailed Returns in Portfolio Optimization via Bayesian VAR and Elliptical Black-Litterman — The BAVAR-BLED algorithm integrates Bayesian VAR and Elliptical Black-Litterman to robustly optimize portfolios against market regime changes and heavy-tailed returns.
• Why AI Still Can’t Solve Your Real Mathematical Optimization Problem — ORPilot's sequential, question-first AI agent approach bridges the gap between LLMs and real-world optimization problems.
• The Shadow Price of Reasoning: Economic Perspective on Optimal Budget Allocation for LLMs — Optimal LLM inference budget allocation can be modeled economically using a global shadow price to maximize accuracy under resource scarcity.
• Training a Trading Agent Using Reinforcement Learning: Reality vs Theory — Reinforcement Learning excels in games but struggles in non-stationary financial markets due to data scarcity and complex reward design.
• sGPO: Trading Inference FLOPs for Training Efficiency in RLVR — sGPO uses cheap inference to profile query difficulty, optimizing RLVR training by reducing wasted FLOPs.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by stat.ML updates on arXiv.org.