Geometric 2D Scene Graph Generation

2026-07-27 · Source: cs.CV updates on arXiv.org · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Robotics & Autonomous Systems · Depth: Expert, extended

Summary

A novel method for Geometric 2D Scene Graph Generation is introduced, focusing on representing assembly relationships between components without relying on semantic data and capable of handling very small datasets. The approach employs a three-step pipeline: first, a Faster R-CNN extracts geometric features and bounding boxes; second, a transformer architecture predicts an adjacency matrix based purely on geometric relationships; and third, a Siamese network with an attentional graph convolutional network (aGCN) characterizes connections, predicting relation labels, permutations, and assembly sequences. Validated on a small dataset of toy model components (train, boat, plane), the method was tested on an unseen car model. Training data augmentation with geometric primitives, including COCO shape templates and abstract shapes, was crucial for generalization, especially given the limited dataset of approximately 50 distinct instances per component. The full model operates efficiently on a single Nvidia RTX 3060 GPU.

Key takeaway

For robotics engineers or production planners developing automated assembly systems with limited labeled data, this geometric scene graph generation method offers a viable solution. You can infer component relationships and assembly sequences purely from geometric cues, even for unseen parts. Consider its three-step architecture for robust structural prediction, but be mindful that inaccuracies in the initial adjacency matrix can propagate to subsequent label and sequence predictions.

Key insights

A three-step pipeline generates geometric scene graphs for component assembly without semantic data, enabling generalization from small datasets.

Principles

• Separate structural inference from semantic reasoning.
• Augment small datasets with geometric primitives.
• Use focal loss for class imbalance.

Method

A Faster R-CNN extracts geometric features, a transformer predicts an adjacency matrix, then a Siamese network with aGCN characterizes relationships, permutations, and sequences.

In practice

• Apply to robotics for assembly understanding.
• Use for consumer product assembly instructions.
• Augment training data with abstract shapes.

Topics

• Scene Graph Generation
• Geometric Deep Learning
• Transformer Networks
• Robotic Assembly
• Data Augmentation
• Siamese Networks

Best for: Research Scientist, AI Scientist, Machine Learning Engineer, Robotics Engineer

Related on AIssential

• Geometric 2D Scene Graph Generation — A novel geometric scene graph generation method characterizes assembly relationships without semantic data, suitable for small datasets.
• Geometry-Guided Modeling of Foundation Features Enables Generalizable Object Shape Deformation Learning — A geometry-guided deformation framework improves 3D object shape recovery by adapting templates to diverse views and novel categories.
• PARSE: Part-Aware Relational Spatial Modeling — Part-level interactions are crucial for resolving spatial ambiguities and generating physically consistent 3D scenes.
• Can We Build Scene Graphs, Not Classify Them? FlowSG: Progressive Image-Conditioned Scene Graph Generation with Flow Matching — FlowSG progressively generates scene graphs by jointly denoising discrete semantics and continuous geometry via hybrid flow matching.
• Robust and Explainable 3D Mode Shape Recognition Using Region-Aware Graph Neural Networks — A canonical graph representation enables robust, explainable, and transferable 3D mode shape recognition.
• A better way to turn 2D designs into 3D models for rapid prototyping — GIFT enables AI models to self-improve 2D-to-3D CAD generation by learning from their own "near-miss" errors.

See Counsel's argued verdicts on the open AI decisions leaders are weighing →

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by cs.CV updates on arXiv.org.