Differentiable Clone-Structured Causal Graphs for End-to-End Cognitive Map Learning from Image Sequences

2026-07-14 · Source: Machine Learning · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Robotics & Autonomous Systems · Depth: Expert, quick

Summary

The paper introduces gradCSCG, a fully differentiable reformulation of the Clone-Structured Causal Graph (CSCG) algorithm, a normative hippocampus model. CSCG traditionally learns interpretable maps from aliased observations but requires a discrete alphabet and is incompatible with neural networks. gradCSCG overcomes this by integrating with a learned vector-quantized variational autoencoder (VQ-VAE) perceptual front-end, enabling end-to-end processing of raw image sequences. This integration uses a soft emission forward pass to allow map-learning objectives to flow into perception, supported by loss-balancing mechanisms to prevent module collapse during joint training. Experiments confirm gradCSCG reproduces CSCG's results on symbolic grid worlds, recovering room topology from heavily aliased observations. It also robustly recovers maps from MNIST image sequences, achieving high edge precision and recall in four heavily aliased environments directly from visual input, demonstrating CSCG's potential as a deep learning building block.

Key takeaway

For AI Scientists developing agents that build structured world models from raw sensory input, gradCSCG offers a robust solution. You should consider integrating this differentiable Clone-Structured Causal Graph with a VQ-VAE front-end to enable end-to-end learning from image sequences. This approach helps your models uncover underlying environmental topologies with high precision, even from heavily aliased observations, simplifying interpretable cognitive map creation.

Key insights

The gradCSCG algorithm enables end-to-end cognitive map learning from raw image sequences by making the normative CSCG model differentiable.

Principles

• Differentiable models enable end-to-end learning.
• Loss balancing prevents module collapse in joint training.
• Cognitive maps can be learned from aliased observations.

Method

Reformulate CSCG as a differentiable module (gradCSCG), couple it with a VQ-VAE perceptual front-end, and use a soft emission forward pass with loss-balancing for joint training.

In practice

• Integrate gradCSCG for visual environment mapping.
• Apply VQ-VAE for perceptual front-end processing.
• Use loss-balancing to stabilize joint deep learning.

Topics

• Differentiable Causal Graphs
• Cognitive Mapping
• VQ-VAE
• End-to-End Learning
• Image Sequence Processing
• Hippocampus Models

Best for: AI Scientist, Research Scientist

Related on AIssential

• Bridging the Gap Between Plausibility and Admissibility: Constraint-Aware Flow Maps for Dynamic Graph Systems — Integrating post-sampling symbolic constraints significantly improves structural admissibility in generative graph trajectory models.
• Dual-Graph Morphing: Cool Multi-Modal AI Agents (Video, Audio) — Graph structures enable AI to dynamically adapt to complex multimodal problems and agentic execution.
• Multi-Level Causal Embeddings — Causal embeddings generalize abstraction, mapping multiple detailed models into a coarser causal model's sub-systems.
• Neural sampling from cognitive maps enables goal-directed imagination and planning — A neural model integrates cognitive maps, stochasticity, and compositional computing for energy-efficient, flexible, goal-directed imagination and planning.
• TERRA: Task-Embedded Reasoning and Representation Architecture for Cross-Domain Applications — TERRA proposes a formal theory and testable hypothesis for cross-domain transfer of structured-state representations and predictors.
• Deep Gaussian Processes on Directed Acyclic Graphs — Deep Gaussian Processes on DAGs effectively model complex compositional functions with robust uncertainty and interpretability.

See Counsel's argued verdicts on the open AI decisions leaders are weighing →

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Machine Learning.