When Do Graph Foundation Models Transfer? A Data-Centric Theory

2026-05-28 · Source: Machine Learning · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Mathematics & Computational Sciences · Depth: Expert, quick

Summary

Graph foundation models (GFMs) often exhibit uneven or negative transfer when applied across different graph domains, despite their goal of reusing a single backbone. This research investigates the data-centric properties that dictate how a fixed representation model's outputs shift between domains. Utilizing a graphon-based continuous limit for dense graphs, the study demonstrates that for both set-based and message-passing tokenizations, any Lipschitz backbone allows an explicit decomposition of cross-domain output shift. This decomposition comprises graph-specific finite-sample approximation terms and an intrinsic, relabeling-invariant domain discrepancy reflecting structural mismatch. A critical component is positional-encoding (PE) stability, with the work establishing stability guarantees for spectral PEs and highlighting distinct behaviors between eigenvector- and subspace-based PEs. Experiments on synthetic and real graphs confirm the theory and offer guidance for data curation in GFM transfer.

Key takeaway

For Machine Learning Engineers developing or deploying Graph Foundation Models, understanding the data-centric theory of transfer is critical. You should analyze the intrinsic domain discrepancy and positional encoding stability between source and target graphs to predict transfer success and mitigate negative transfer. This framework provides guidance for curating training data and selecting appropriate PEs, directly impacting your model's cross-domain performance.

Key insights

GFM transfer shift can be explicitly decomposed into finite-sample approximation and intrinsic domain discrepancy, guided by positional encoding stability.

Principles

• GFM transfer behavior is determined by data domain properties.
• Output shift decomposes into finite-sample and domain discrepancy.
• Positional encoding stability is key for GFM transfer.

Method

The study employs a graphon-based continuous limit for dense graphs to derive an explicit decomposition of cross-domain output shift for Lipschitz backbones.

In practice

• Guide data curation for improved GFM transfer.
• Analyze domain discrepancy to predict transfer success.
• Evaluate PE stability for GFM architecture design.

Topics

• Graph Foundation Models
• Transfer Learning
• Data-Centric AI
• Graphons
• Positional Encoding
• Domain Discrepancy

Best for: Research Scientist, AI Scientist, Machine Learning Engineer

Related on AIssential

• Transfer Learning from Foundational Optimization Embeddings to Unsupervised SAT Representations — Foundational optimization embeddings transfer to SAT problems, enabling unsupervised structural analysis.
• How to Actually Train a Foundation Model — The model's quality is primarily determined by data distribution construction, filtering, weighting, and auditing, not just architecture and scale.
• A Closed-Form Persistence-Landmark Pipeline for Certified Point-Cloud and Graph Classification — PLACE offers a closed-form pipeline for point cloud and graph classification with certified guarantees from persistent homology.
• Uncovering Symmetry Transfer in Large Language Models via Layer-Peeled Optimization — LLM training implicitly transfers data symmetries into structured geometric configurations within model parameters.
• A Mechanistic Analysis of Sim-and-Real Co-Training in Generative Robot Policies — Effective co-training balances cross-domain representation alignment with domain discernibility for adaptive robot policy transfer.
• Format-Constraint Coupling in Knowledge Graph Construction from Statistical Tables — Serialization format and schema constraints interact critically in knowledge graph construction from statistical tables, impacting fidelity.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Machine Learning.