Interventional Time Series Priors for Causal Foundation Models

2026-03-13 · Source: cs.LG updates on arXiv.org · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Data Science & Analytics · Depth: Advanced, long

Summary

CausalTimePrior is a novel framework designed to generate synthetic temporal structural causal models (TSCMs) with paired observational and interventional time series, addressing a critical gap in training causal foundation models for time series data. Existing benchmarks provide observational data but lack the interventional targets necessary for learning causal effects. CausalTimePrior supports configurable causal graph structures, nonlinear autoregressive mechanisms, regime-switching dynamics, and multiple intervention types, including hard, soft, and time-varying interventions. A proof-of-concept PFN trained on CausalTimePrior demonstrated in-context causal effect estimation on held-out TSCMs, achieving comparable RMSE to per-dataset VAR baselines without requiring per-sample fitting. The framework generates diverse TSCMs, including 15% with regime-switching dynamics, and maintains a 0% divergence rate across 100K samples, ensuring data quality and stability.

Key takeaway

For research scientists developing foundation models for time series causal inference, CausalTimePrior offers a crucial synthetic data generation framework. You should leverage this tool to create diverse, interventional time series datasets, enabling the training of models that can perform in-context causal effect estimation without task-specific fine-tuning. This approach helps overcome the current limitation of observational-only benchmarks, accelerating the development of robust causal AI systems capable of reasoning about interventions in dynamic environments.

Key insights

CausalTimePrior generates synthetic interventional time series data to train foundation models for time series causal inference.

Principles

• Interventional data is crucial for training causal foundation models.
• Diverse intervention types improve causal reasoning.
• Regime-switching dynamics enhance real-world applicability.

Method

The CausalTimePrior method samples TSCMs with time-lagged DAGs, nonlinear structural equations, and diverse noise, then generates paired observational and interventional time series by modifying structural equations for various intervention types.

In practice

• Use CausalTimePrior to generate synthetic data for causal PFN training.
• Incorporate regime-switching dynamics for robust causal models.
• Train models on diverse intervention types for better generalization.

Topics

• Causal Foundation Models
• Time Series Causal Inference
• Synthetic Data Generation
• Interventional Data
• Regime-Switching Dynamics

Code references

• thummd/CausalTimePrior

Best for: Research Scientist, AI Researcher, AI Scientist, Machine Learning Engineer

Related on AIssential

• Benchmarking Counterfactual Prediction in Epidemic Time Series with Time-Varying Interventions — A new benchmark provides realistic counterfactuals for epidemic time series, revealing challenges in causal inference methods.
• The brewing GenAI data science revolution — Foundation models are extending beyond text/image to transform tabular and time-series data analysis.
• Causal Semantic Alignment for LLM-based Time Series Forecasting — CVAformer disentangles time series variables into invariant and dynamic components to improve LLM-based forecasting accuracy by mitigating confounding effects.
• Exploring the Potential of Probabilistic Transformer for Time Series Modeling: A Report on the ST-PT Framework — Probabilistic Transformer (PT) maps Transformer operations to Mean-Field Variational Inference on a Conditional Random Field (CRF).
• Interview with AAAI Fellow Yan Liu: machine learning for time series — Physics-informed foundation models enable accurate, adaptable time series analysis, especially in data-scarce scientific and real-world applications.
• CausalPOI: Spatio-Temporal Graph-Based Causal Modeling for Cold-Start POI Check-in Forecasting — CausalPOI forecasts cold-start POI check-ins by modeling causal effects through spatio-temporal functional interaction graphs.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by cs.LG updates on arXiv.org.