Interview with Sukanya Mandal: Synthesizing multi-modal knowledge graphs for smart city intelligence

2026-04-09 · Source: ΑΙhub · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Data Science & Analytics, Internet of Things (IoT) & Connected Devices · Depth: Advanced, short

Summary

Sukanya Mandal and Noel O’Connor introduced LLMasMMKG, a four-stage framework that uses large language models (LLMs) to automate the creation of synthetic multi-modal knowledge graphs (MMKGs) for smart city cognitive digital twins (CDTs). CDTs are AI-enabled virtual replicas that model dynamic urban systems, reasoning over integrated insights from diverse sources like traffic sensors, healthcare, and social media to anticipate issues. The LLMasMMKG framework gathers and preprocesses heterogeneous data, unifies it using Sentence-BERT embeddings, and employs fine-tuned BERT for entity recognition and GPT-4 for relationship extraction. A key innovation is LLM-driven synthetic data generation to address data sparsity, privacy risks, and biases. The process culminates in an RDF-formatted knowledge graph with a hierarchical domain ontology, demonstrated across smart home, healthcare, transportation, and energy domains.

Key takeaway

For AI Scientists and Machine Learning Engineers developing smart city solutions, the LLMasMMKG framework offers a scalable approach to overcome data scarcity and privacy concerns. You should consider integrating LLM-driven synthetic data generation and multi-modal knowledge graphs to build robust cognitive digital twins. This method reduces manual effort in KG construction and enables more comprehensive urban system reasoning, supporting sustainable decision-making.

Key insights

LLMasMMKG uses LLMs to automate synthetic multi-modal knowledge graph creation for smart city cognitive digital twins.

Principles

• CDTs reason over integrated multi-modal data.
• MMKGs excel at complex urban interdependencies.
• Synthetic data mitigates real-world data issues.

Method

The LLMasMMKG framework involves data gathering, multi-modal representation learning via Sentence-BERT, LLM-guided knowledge extraction (BERT for entities, GPT-4 for relations), and RDF knowledge graph population with a hierarchical ontology.

In practice

• Use GPT-4-turbo for diverse text generation.
• Simulate sensor data with Python (pandas, numpy).
• Employ Sentence-BERT for semantic similarity.

Topics

• Smart City Cognitive Digital Twins
• Multi-modal Knowledge Graphs
• LLM-assisted Knowledge Graph Creation
• Synthetic Data Generation
• BERT and GPT-4

Code references

• sukanyamandal/LLMasMMKG_Code

Best for: AI Scientist, Machine Learning Engineer, Research Scientist

Related on AIssential

• LLMs+Graphs: Toward Graph-Native, Synergistic AI Systems — LLMs require graph integration to overcome structured reasoning limitations and enable graph-native AI systems.
• A Hybrid Framework for Natural Language Querying of IFC Models with Relational and Graph Representations — Combining relational and graph representations with iterative LLM reasoning enhances IFC model accessibility.
• BLINKG: A Benchmark for LLM-Integrated Knowledge Graph Generation — BLINKG benchmarks LLM capabilities in mapping diverse data to ontologies for Knowledge Graph generation.
• Construction of Knowledge Graph based on Language Model — PLMs significantly enhance Knowledge Graph construction by automating entity and relation extraction from text.
• Large Language Models in Transportation Systems Management and Operations: From Text Reasoning to Multi-modal Decision Support — LLMs and MM-LLMs offer a new mechanism for integrating diverse transportation data into operator-facing decision support systems.
• LogosKG: Hardware-Optimized Scalable and Interpretable Knowledge Graph Retrieval — LogosKG offers scalable, interpretable k-hop retrieval for large knowledge graphs via hardware-optimized symbolic operations.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by ΑΙhub.