Construction of Knowledge Graph based on Language Model

2026-04-22 · Source: cs.CL updates on arXiv.org · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Data Science & Analytics, Natural Language Processing · Depth: Expert, long

Summary

This paper reviews recent advancements in Knowledge Graph (KG) construction using Pre-trained Language Models (PLMs) and introduces a new framework called LLHKG for building Hyper-Relational Knowledge Graphs (HRKGs) with lightweight Large Language Models (LLMs). Traditional KG methods often require extensive manual annotation or suffer from weak generalization in deep learning approaches. PLMs, with their language understanding and generation capabilities, can automatically extract entities and relations from text. The LLHKG framework, which utilizes lightweight LLMs like LLama3.1:8B and Qwen2.5:7B, demonstrates HRKG construction capabilities comparable to GPT3.5, achieving a BERTScore only 0.01 lower. The framework includes hyper-relation extraction and a correction module, addressing the limitations of existing LLM-based HRKG methods which often have insufficient accuracy.

Key takeaway

For research scientists developing knowledge graph solutions, you should consider integrating lightweight LLMs into your HRKG construction workflows. The LLHKG framework demonstrates that models like LLama3.1:8B and Qwen2.5:7B can achieve performance on par with larger models like GPT3.5 for HRKG creation, potentially reducing computational costs and increasing accessibility. Explore prompt optimization and iterative correction modules to enhance accuracy and reliability in your implementations.

Key insights

Lightweight LLMs can construct Hyper-Relational Knowledge Graphs with performance comparable to GPT3.5.

Principles

• PLMs excel at extracting entities and relations.
• HRKGs better represent complex many-to-many relations.
• Iterative validation improves KG quality.

Method

The LLHKG framework uses lightweight LLMs for hyper-relation extraction and a correction module to refine semantics and format, leveraging prompt optimization for specific tasks.

In practice

• Use LLama3.1:8B for hyper-relation extraction.
• Employ Qwen2.5:7B for hyper-relation correction.
• Optimize prompts for task-specific knowledge extraction.

Topics

• Knowledge Graph Construction
• Pre-trained Language Models
• Large Language Models
• Hyper-Relational Knowledge Graph
• LLHKG Framework

Best for: Research Scientist, AI Scientist, Machine Learning Engineer, NLP Engineer

Related on AIssential

• Construction of Knowledge Graph based on Language Model — PLMs significantly enhance Knowledge Graph construction by automating entity and relation extraction from text.
• Code-on-Graph: Iterative Programmatic Reasoning via Large Language Models on Knowledge Graphs — Code-on-Graph integrates LLMs with KGs using programmatic reasoning, representing schemas as Python classes for flexible, scalable knowledge utilization.
• Hypergraph as Language — Hyper-Align enables LLMs to natively model high-order hypergraph associations by treating hypergraphs as a structural language.
• Building Trust in the Skies: A Knowledge-Grounded LLM-based Framework for Aviation Safety — Combining LLMs with dynamically updated Knowledge Graphs enhances trustworthiness and verifiability in safety-critical aviation analytics.
• How SHACL Makes Your LLMs Hum — Integrating SHACL-defined knowledge graphs with LLMs significantly reduces hallucinations and improves response accuracy.
• Retrieval Augmented (Knowledge Graph), and Large Language Model-Driven Design Structure Matrix (DSM) Generation of Cyber-Physical Systems — LLMs, especially with RAG and GraphRAG, can automate Design Structure Matrix generation for complex systems.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by cs.CL updates on arXiv.org.