DAG-FM: A Foundation Model for Causal Discovery under Heterogeneous Causal Mechanisms

2026-07-14 · Source: stat.ML updates on arXiv.org · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Data Science & Analytics · Depth: Expert, extended

Summary

DAG-FM is a novel foundation model architecture designed for causal discovery from observational tabular data, specifically addressing challenges posed by heterogeneous causal mechanisms and high-dimensional search spaces. It amortizes causal discovery by decomposing the process into two auto-regressive stages: a leaf-node predictor and a parent-node predictor, both utilizing Transformer-based sub-modules. The model incorporates a robust tabular interaction block for complex row-column interactions and introduces Mixture-of-Leaf-Experts (MoLE) to dynamically adapt to diverse Functional Causal Model (FCM) assumptions, ensuring identifiable mechanism families. Through an iterative inference algorithm, DAG-FM extracts causal orderings and constructs valid Directed Acyclic Graphs (DAGs). Extensive experiments demonstrate its state-of-the-art performance on synthetic benchmarks and real-world datasets like Sachs (7,466 samples, 11 variables) and Causal Chamber (10,000 samples, 38 variables), outperforming traditional algorithms and recent foundation models in accuracy and scalability, while operating within a 24GB VRAM constraint for large datasets.

Key takeaway

For Machine Learning Engineers working on causal discovery with complex, heterogeneous tabular data, you should consider DAG-FM. This foundation model provides state-of-the-art accuracy and scalability for identifying Directed Acyclic Graphs, even when underlying causal mechanisms vary. Its ability to handle diverse functional causal models and operate within 24GB VRAM makes it a practical choice. Evaluate DAG-FM to enhance the robustness and efficiency of your causal inference pipelines.

Key insights

DAG-FM uses a two-stage Transformer and Mixture-of-Leaf-Experts to discover identifiable causal DAGs from heterogeneous tabular data.

Principles

• Decompose DAG identification into leaf and parent prediction.
• Dynamically adapt to heterogeneous causal mechanisms via expert mixtures.
• Design prior spaces to guarantee unique DAG identifiability.

Method

DAG-FM recursively identifies leaf nodes using MoLE, then prunes the DAG by predicting parents for each identified leaf, forming a valid causal graph.

In practice

• Apply to high-dimensional tabular data with mixed causal types.
• Integrate expert priors to guide mechanism family selection.
• Utilize for causal discovery in bioinformatics or manufacturing.

Topics

• Causal Discovery
• Foundation Models
• Directed Acyclic Graphs
• Heterogeneous Mechanisms
• Mixture-of-Experts
• Transformer Architectures

Best for: AI Scientist, Machine Learning Engineer, Research Scientist

Related on AIssential

• Structure Learning on Clustered Data — A new method extends mixed models to DAG structure learning, enabling causal discovery in heterogeneous clustered data.
• CausalMoE: A Billion-Scale Multimodal Foundation Model for Granger Causal Discovery with Pattern-Routed Heterogeneous Experts — CausalMoE uses pattern-routed heterogeneous experts and multimodal priors for robust Granger Causal Discovery in complex, dynamic time series.
• DAGForge: Auditable Causal DAG Authoring with Biomedical Literature — DAGForge automates auditable causal DAG construction for biomedical analysis by linking LLM-generated judgments to verifiable literature evidence.
• DKCD: Domain Knowledge-Enhanced Causal Discovery from Unstructured Data — Integrating domain knowledge graphs with LLMs significantly enhances causal discovery from unstructured data in high-expertise fields.
• polyDAG: Polynomial Acyclicity Constraints for Efficient Continuous Causal Discovery in Visual Semantic Graphs — polyDAG introduces a polynomial trace constraint for efficient continuous causal discovery in visual semantic graphs.
• CDFM: Towards a General-Purpose Causal Discovery Foundation Model — CDFM unifies causal discovery by learning generalizable representations from diverse mechanisms for zero-shot structural inference.

See Counsel's argued verdicts on the open AI decisions leaders are weighing →

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by stat.ML updates on arXiv.org.