When Valid Signals Fail: Regime Boundaries Between LLM Features and RL Trading Policies

2026-06-25 · Source: Paper Index on ACL Anthology · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Data Science & Analytics, Capital Markets & Investment Management · Depth: Expert, quick

Summary

A study explores the efficacy of large language models (LLMs) in generating continuous numerical features for reinforcement learning (RL) trading agents. Researchers developed a modular pipeline where a frozen LLM acts as a stateless feature extractor, converting daily news and filings into fixed-dimensional vectors for a downstream PPO agent. A novel automated prompt-optimization loop was introduced, tuning the extraction prompt directly against the Information Coefficient (Spearman rank correlation between predicted and realized returns) instead of traditional NLP losses. This method successfully discovered genuinely predictive features, achieving an Information Coefficient above approximately 0.15 on held-out data. However, these valid LLM-derived features did not consistently improve downstream task performance; during a distribution shift caused by a macroeconomic shock, the augmented agent underperformed a price-only baseline. While the agent recovered in calmer test regimes, macroeconomic state variables proved to be the most robust drivers of policy improvement, revealing a critical gap between feature-level validity and policy-level robustness.

Key takeaway

For Machine Learning Engineers developing RL trading agents, you should critically evaluate LLM-derived features beyond initial predictive validity. While prompt-optimized LLM features can achieve an Information Coefficient above 0.15, your models may underperform during macroeconomic shocks if solely reliant on them. Prioritize integrating robust macroeconomic state variables into your policy design to ensure resilience against distribution shifts, rather than expecting LLM features alone to maintain performance across all market regimes.

Key insights

LLM-derived trading features can be predictive but fail under macroeconomic distribution shifts, impacting policy robustness.

Principles

• Feature validity does not guarantee policy robustness.
• Distribution shifts challenge LLM-augmented RL agents.
• Macroeconomic state variables offer robust policy drivers.

Method

An automated prompt-optimization loop tunes LLM feature extraction prompts against Information Coefficient (IC) for RL trading agents, bypassing NLP losses.

In practice

• Use IC for prompt optimization in financial LLMs.
• Test LLM features under diverse market regimes.
• Incorporate macroeconomic data for robust policies.

Topics

• Large Language Models
• Reinforcement Learning
• Algorithmic Trading
• Feature Engineering
• Prompt Optimization
• Distribution Shift

Best for: AI Scientist, Machine Learning Engineer, Research Scientist

Related on AIssential

• Is Capability a Liability? More Capable Language Models Make Worse Forecasts When It Matters Most — More capable LLMs make worse forecasts for superlinear growth and tail risk, especially in the upper tail.
• Agentic Trading: When LLM Agents Meet Financial Markets — LLM-based trading agent research lacks comparable evaluation protocols, hindering reliable performance assessment and reproducibility.
• Does RL Expand the Capability Boundary of LLM Agents? A PASS@(k,T) Analysis — RL expands LLM agent capabilities for compositional tool use, unlike its role in static reasoning.
• Heteroskedastic Signals in Budgeted LLM Verification: Structural Heterogeneity Limits Optimization Gains — The global signal comparability assumption in LLM uncertainty is flawed due to heteroskedastic quality across cost strata, limiting optimization gains.
• The signal is the ceiling: Measurement limits of LLM-predicted experience ratings from open-ended survey text — Input text characteristics limit LLM prediction accuracy more than prompt or model choice.
• Generalization in LLM Problem Solving: The Case of the Shortest Path — LLMs generalize spatially but fail at length scaling in shortest-path problems due to recursive instability.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Paper Index on ACL Anthology.