Construction of Historical Knowledge Graphs Based on BERT and Graph Neural Networks

2026-06-01 · Source: Artificial Intelligence · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Natural Language Processing, Knowledge Representation · Depth: Expert, quick

Summary

A new high-level architecture combines Bidirectional Encoder Representations of Transformers (BERT) and Graph Neural Networks (GNN) to construct historical knowledge graphs. This system extracts entities and relationships from diverse historical texts, including municipal records, parliamentary documents, and historical correspondence. It systematically addresses challenges like linguistic ambiguities, context-limited references, and inconsistent grammatical norms inherent in traditional historical documents. The joint BERT-GNN system demonstrated superior performance, achieving higher Precision, Recall, and F1-scores (Table 2) compared to conventional rule-based techniques and other deep-learning baselines. This architecture effectively handles complex nested structures and implicit reference issues, proving that combining relational graph learning algorithms with context-sensitive semantic representation techniques can automatically extract historical data for knowledge repositories. The study also develops an image retrieval system based on FastRQNet and Vilt-qaformer+RoBInet.

Key takeaway

For NLP Engineers and Research Scientists tasked with constructing knowledge graphs from complex historical texts, you should consider adopting a joint BERT-GNN architecture. This approach demonstrably improves Precision, Recall, and F1-scores over traditional methods, enabling more accurate extraction of entities and relationships from ambiguous and context-limited documents. Implementing such a system can significantly enhance the quality and completeness of your historical knowledge repositories, even handling nested structures and implicit references effectively.

Key insights

Combining BERT and GNNs effectively extracts entities and relationships from complex historical texts for knowledge graph construction.

Principles

• Relational graph learning enhances historical data extraction.
• Context-sensitive semantics improve entity-relationship extraction.
• Joint BERT-GNN outperforms rule-based and deep learning baselines.

Method

The proposed method integrates BERT for context-sensitive semantic representation with GNNs for relational graph learning. This joint system extracts entities and relationships from historical texts, addressing linguistic ambiguities and implicit references.

In practice

• Apply BERT-GNN for historical document analysis.
• Use joint models for complex text structures.
• Develop knowledge graphs from unstructured archives.

Topics

• Historical Knowledge Graphs
• BERT
• Graph Neural Networks
• Entity-Relationship Extraction
• Digital Humanities
• Image Retrieval Systems

Best for: AI Scientist, NLP Engineer, Research Scientist

Related on AIssential

• From Gates to Boundaries — An event-based graph model with SHACL and ODRL forms an active "holon boundary" for dynamic knowledge graph governance.
• Build Meaning Before Machines: Why Semantics, Ontologies, And Knowledge Graphs Matter For Agentic AI — Agentic AI requires explicit data meaning, making semantic layers and knowledge graphs essential architectural components.
• Writing Business Rules in SHACL — SHACL's "sh:sparql" enables declarative, contextual business rules by inverting SPARQL query logic to detect violations or conditions.
• GJDNet: Robust Graph Neural Networks via Joint Disentangled Learning Against Adversarial Attacks — GJDNet enhances GNN robustness by jointly disentangling node representations and decision spaces against adversarial attacks.
• CEON: Circular Economy Ontology Network — CEON provides a cross-sectorial ontology network to enable semantic interoperability and data documentation for circular economy strategies.
• Absorbing Complexity: An Interaction-Native Knowledge Harness for Financial LLM Agents — Financial AI adoption is driven by systems absorbing complexity rather than transferring it to users.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Artificial Intelligence.