ATGBuilder: Feature-Assisted Graph Learning for Activity Transition Graph Construction with Seed Supervision

2026-06-26 · Source: cs.SE updates on arXiv.org · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Software Development & Engineering · Depth: Advanced, extended

Summary

AtgBuilder is a novel feature-assisted graph-learning approach designed for seed-supervised Activity Transition Graph (ATG) construction in Android applications. It addresses limitations of traditional static and dynamic analysis by using a Large Language Model (GPT-4o) to summarize UI activity metadata into compact textual functionality summaries, which are then encoded using a sentence-embedding model. Crucially, AtgBuilder explicitly models widget-trigger information as edge attributes, incorporating an auxiliary widget-attribute reconstruction objective during training. Evaluated on a 98-app benchmark with manually-checked ground-truth ATGs, AtgBuilder improved the F1-score by 15.41% to 77.34% over SOTA baselines. Furthermore, its predicted ATGs enhanced activity and transition coverage when used as navigation guidance for automated GUI-exploration tools like Monkey, APE, and FastBot2.

Key takeaway

For Machine Learning Engineers or AI Scientists building robust mobile app analysis tools, AtgBuilder offers a significant advancement in Activity Transition Graph construction. You should consider integrating LLM-driven semantic feature extraction and auxiliary widget-attribute modeling into your graph learning pipelines. This approach demonstrably improves ATG quality and enhances the effectiveness of automated GUI exploration, leading to better coverage and more reliable app analysis.

Key insights

AtgBuilder uses LLM-summarized UI functionality and widget-trigger edge attributes to predict Android app activity transitions.

Principles

• Prioritize functionality summaries over raw UI layouts for activity representation.
• Model widget-trigger information as activity-independent edge attributes.
• Employ auxiliary reconstruction objectives to preserve critical feature information.

Method

AtgBuilder extracts static UI metadata, transforms activity/widget info into fixed-size node/edge attributes via LLM summaries and embeddings, then trains a GNN-based link predictor with an auxiliary widget-attribute reconstruction objective.

In practice

• Utilize LLMs (e.g., GPT-4o) for semantic UI layout summarization.
• Integrate widget-trigger data as distinct edge features in graph models.
• Implement validation-based thresholding for robust link prediction.

Topics

• Activity Transition Graphs
• Graph Neural Networks
• Large Language Models
• Android App Analysis
• GUI Testing Automation
• Link Prediction

Code references

• libre-tube/LibreTube
• TeamNewPipe/NewPipe

Best for: Research Scientist, AI Scientist, Machine Learning Engineer, AI Engineer

Related on AIssential

• BLINKG: A Benchmark for LLM-Integrated Knowledge Graph Generation — BLINKG provides a standardized benchmark to evaluate Large Language Models' effectiveness in mapping data schemes to ontology concepts for Knowledge Graph generation.
• Learning Long Range Spatio-Temporal Representations over Continuous Time Dynamic Graphs with State Space Models — CTDG-SSM unifies state-space modeling with graph topology to capture long-range spatio-temporal dependencies in continuous-time dynamic graphs.
• SkillDAG: Self-Evolving Typed Skill Graphs for LLM Skill Selection at Scale — Dynamic, graph-based structural retrieval significantly enhances LLM agent skill selection and performance over static methods.
• LLMs+Graphs: Toward Graph-Native, Synergistic AI Systems — LLMs require graph integration to overcome structured reasoning limitations and enable graph-native AI systems.
• LogosKG: Hardware-Optimized Scalable and Interpretable Knowledge Graph Retrieval — LogosKG offers scalable, interpretable k-hop retrieval for large knowledge graphs via hardware-optimized symbolic operations.
• GraphScout: Empowering Large Language Models with Intrinsic Exploration Ability for Agentic Graph Reasoning — GraphScout enables LLMs to autonomously explore knowledge graphs, internalizing reasoning through synthesized training data.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by cs.SE updates on arXiv.org.