Evaluating Large Language Model News Sentiment in Finance under Liquidity and Market Frictions

2026-06-25 · Source: Paper Index on ACL Anthology · Field: Finance & Economics — Capital Markets & Investment Management, FinTech & Digital Financial Services, Artificial Intelligence & Machine Learning · Depth: Expert, medium

Summary

A study evaluates large language models' ability to extract useful sentiment signals from firm-specific financial news, accounting for realistic market frictions. Analyzing 3,129,924 U.S. news items from January 2010 to January 2026, filtered to 973,481 tradable items for 3,452 firms, the research compares LLaMA–3, OPT, RoBERTa, BERT, FinBERT, and the Loughran–McDonald dictionary. LLaMA–3 demonstrated the strongest classification performance with 78.2% accuracy and the largest predictive coefficients in panel regressions. When assessed through daily rebalanced long–short portfolios with a 5 bps trading cost, the LLaMA–3 strategy generated an approximate 180% cumulative return from June 2024 to January 2026, significantly outperforming OPT (155%), RoBERTa (120%), and the dictionary-based strategy, which lost 9%. This highlights that high-capacity language models maintain economically meaningful predictive content even under real-world trading constraints.

Key takeaway

For quantitative analysts and machine learning engineers developing financial trading strategies, you should prioritize evaluating sentiment models beyond mere offline accuracy. Incorporate realistic market frictions like a 5 bps trading cost and liquidity constraints into your backtesting. High-capacity models like LLaMA–3 offer significantly superior predictive content and cumulative returns compared to simpler lexicon-based approaches, making them a more robust choice for implementable portfolio performance.

Key insights

High-capacity LLMs extract economically meaningful financial news sentiment even under realistic market frictions, unlike simpler lexicon methods.

Principles

• Financial NLP evaluation needs realistic market frictions.
• Offline accuracy alone is insufficient for utility.
• High-capacity LLMs outperform lexicon methods in real trading.

Method

The study evaluates sentiment models using classification performance, return predictability, and implementable portfolio performance with a 5 bps trading cost, after filtering news for tradable stocks and novelty.

In practice

• Use LLaMA–3 for financial news sentiment analysis.
• Incorporate 5 bps trading costs in backtesting.
• Filter news for tradable stocks and novelty.

Topics

• Large Language Models
• Financial News Sentiment
• Market Frictions
• Algorithmic Trading
• LLaMA–3
• Quantitative Finance

Best for: AI Engineer, NLP Engineer, Research Scientist, AI Scientist, Machine Learning Engineer, Data Scientist

Related on AIssential

• CSTrader: A Testbed for Language-Grounded Trading in a Community-Driven Virtual Asset Market — Niche, language-driven markets offer a robust benchmark for language-to-action LLM research.
• Enhancing Brazilian Inflation Forecasts through Sentiment Analysis Using Large Language Models — LLM-derived sentiment from policy minutes significantly enhances inflation forecasting accuracy when aligned with human assessments.
• Fin-RATE: A Real-world Financial Analytics and Tracking Evaluation Benchmark for LLMs on SEC Filings — LLMs struggle with multi-document, multi-entity, and temporal financial reasoning, revealing critical gaps in current evaluation.
• The Math Behind the Bots Dominating Prediction Markets — Strict mathematical models, not sentiment, drive consistent profitability in prediction markets.
• Introduction to quantitative finance Part 26: Whether to use forecasting methods, or to tell an LLM… — Forecasting in quantitative finance faces challenges with LLMs, necessitating memory-enhanced reward shaping for consistency.
• Beyond Polarity: Multi-Dimensional LLM Sentiment Signals for WTI Crude Oil Futures Return Prediction — Multi-dimensional LLM sentiment, beyond polarity, improves WTI crude oil futures return prediction.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Paper Index on ACL Anthology.