Anomaly detection on financial data — A classical ML based approach

2026-07-20 · Source: Machine Learning on Medium · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Data Science & Analytics, FinTech & Digital Financial Services · Depth: Intermediate, medium

Summary

An unsupervised anomaly detection system, leveraging a classical LSTM encoder-decoder model, was developed to identify anomalous behavior in simulated corporate financial time series data. This system aims to flag deviations from normal patterns early for banking and regulatory systems. The LSTM model, trained exclusively on ordinary historical data, uses prediction error as an anomaly score, effectively identifying unusual data without requiring labeled "bad" examples. Key challenges addressed include managing rare anomalies by capping alerts to investigator capacity, which boosted precision from 12% to 60%. Additionally, robust pre-filtering of streaming input data is crucial to prevent model corruption from incomplete or malformed records. Success is measured by recall and precision, avoiding misleading overall accuracy metrics in rare event scenarios.

Key takeaway

For Data Scientists or MLOps Engineers deploying anomaly detection for financial time series, you must prioritize robust data quality checks and calibrate alert thresholds to human investigative capacity, rather than relying solely on misleading overall accuracy metrics. This strategy, which improved precision from 12% to 60% in experiments, prevents alert fatigue and ensures that the system delivers actionable insights from genuinely rare events.

Key insights

Anomaly detection on time series benefits from models that learn normal patterns and contextual history.

Principles

• Time series anomaly detection requires models with memory for historical context.
• For rare anomalies, cap alerts to human investigative capacity for higher precision.
• Pre-process streaming data rigorously to prevent model corruption and false alerts.

Method

An LSTM encoder-decoder is trained solely on normal time series data to predict future values; its prediction error then serves as the anomaly score.

In practice

• Employ LSTMs for contextual time series anomaly detection.
• Set a fixed maximum number of daily anomaly alerts.
• Implement strict data validation before model ingestion.

Topics

• Anomaly Detection
• Financial Data
• Time Series Analysis
• LSTM
• Data Quality
• Unsupervised Learning

Code references

• MAY2704/anomaly_detection

Best for: Machine Learning Engineer, Data Scientist, MLOps Engineer

Related on AIssential

• A Comprehensive, Practical Guide to Unsupervised Anomaly Detection — Unsupervised anomaly detection models normal behavior to identify rare, unknown deviations.
• Why Powerful Machine Learning Is Deceptively Easy — Methodological rigor in ML is crucial to distinguish genuine model performance from illusory gains.
• ChronosAD: Leveraging Time Series Foundation Models for Accurate Anomaly Detection — ChronosAD leverages time series foundation models and a temporal block for robust, generalized anomaly detection with minimal tuning.
• Beyond Defensive Reporting: Machine Learning for Active Anti-Money Laundering Control in Insurance — Machine learning, particularly with fraud data, can effectively detect insurance money laundering pre-payout.
• PAI: Preserving Amplitude Information in Representation-Based Time-Series Anomaly Detection — Representation-based time-series anomaly detection benefits significantly from explicitly preserving amplitude information, which PAI achieves through a novel scoring scheme.
• Rarity-Gated Context Conditioning for Offline Imitation Learning-Based Maritime Anomaly Detection — Explicitly accounting for context rarity reduces false alarms and improves robustness in contextual anomaly detection.

See Counsel's argued verdicts on the open AI decisions leaders are weighing →

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Machine Learning on Medium.