Graph In-Context Operator Networks for Generalizable Spatiotemporal Prediction

2026-04-16 · Source: cs.AI updates on arXiv.org · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Data Science & Analytics, Environmental Science & Earth Systems · Depth: Expert, extended

Summary

A new model, Graph In-Context Operator Network (GICON), has been developed to systematically compare in-context operator learning against classical single-operator learning for real-world spatiotemporal prediction tasks. GICON integrates graph message passing for geometric generalization with an example-aware positional encoding for cardinality generalization, allowing it to handle irregularly sampled physical data and scale robustly from few training examples (0-5) to 100 at inference. Experiments on air quality prediction across two Chinese regions (Beijing-Tianjin-Hebei and Yangtze River Delta) using 70,128 hours of observations from 2016-2023 demonstrate that in-context operator learning, particularly with operator diversity, outperforms classical single-operator learning on complex tasks, showing improved generalization across spatial domains and stable performance with increasing example counts.

Key takeaway

For AI Scientists and Machine Learning Engineers developing spatiotemporal prediction models, GICON offers a robust framework that leverages in-context learning. You should prioritize training with operator diversity, as this significantly improves generalization and example utilization, especially for complex tasks and out-of-distribution scenarios. Consider adopting GICON's graph-based approach for systems with irregular geometries, like environmental monitoring networks, to enhance model adaptability and performance.

Key insights

In-context operator learning with diverse operators significantly outperforms classical methods for complex spatiotemporal prediction.

Principles

• Operator diversity enhances in-context learning generalization.
• Graph message passing enables geometric generalization.
• Content-aware positional encoding supports cardinality generalization.

Method

GICON combines graph message passing for spatial updates with per-node transformer-based in-context learning, using example-aware attention biases and key-value distinction offsets for positional encoding.

In practice

• Use GICON for air quality forecasting on irregular sensor networks.
• Train with diverse operators for better out-of-distribution generalization.
• Employ FAISS for efficient retrieval of contextual examples.

Topics

• Graph Neural Networks
• In-Context Learning
• Operator Learning
• Spatiotemporal Prediction
• Air Quality Forecasting

Code references

• KellerJordan/Muon

Best for: AI Scientist, Machine Learning Engineer, Research Scientist

Related on AIssential

• Exploitation of Hidden Context in Dynamic Movement Forecasting: A Neural Network Journey from Recurrent to Graph Neural Networks and General Purpose Transformers — Hybrid LSTM models with contextual data excel in dynamic trajectory prediction, outperforming other ML architectures.
• Robust Representation Learning through Explicit Environment Modeling — Explicitly modeling and marginalizing environmental variation yields more robust OOD generalization than invariance-seeking methods.
• GITCO: Gated Inference-Time Context Optimization in TSFMs — Patch-based TSFMs can be improved at inference time by optimizing input context to suppress harmful patches.
• From XXLTraffic to EvoXXLTraffic: Scaling Traffic Forecasting to Sensor-Evolving Networks — Existing traffic forecasting benchmarks fail to account for real-world sensor network evolution, leading to unrealistic SOTA performance.
• Spatio-Temporal Models in AI: Teaching AI to Understand Space and Time — Spatio-Temporal AI models learn interconnected changes across space and time, crucial for dynamic real-world predictions.
• Pretrain Where? Investigating How Pretraining Data Diversity Impacts Geospatial Foundation Model Performance — Geographic pretraining data composition significantly impacts geospatial foundation model performance, with spectral diversity being a key driver.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by cs.AI updates on arXiv.org.