Code-on-Graph: Iterative Programmatic Reasoning via Large Language Models on Knowledge Graphs

2026-06-02 · Source: Artificial Intelligence · Field: Technology & Digital — Artificial Intelligence & Machine Learning · Depth: Expert, quick

Summary

Code-on-Graph (CoG) is a novel programmatic reasoning framework designed to integrate Large Language Models (LLMs) with Knowledge Graphs (KGs), addressing LLM limitations like outdated knowledge and hallucinations. Existing LLM-KG integration methods suffer from inflexible predefined operators and unscalability due to direct factual knowledge injection into prompts. CoG overcomes these by identifying KG schemas from retrieved facts, representing them as Python classes, and generating executable code grounded in these classes. During execution, retrieved facts are instantiated as objects, enabling flexible code-based reasoning without injecting large-scale knowledge directly into prompts. This approach allows for more complex semantic capture and better scalability. Experiments on WebQSP, CWQ, and GrailQA benchmarks demonstrate that CoG outperforms prior state-of-the-art models by up to 10.5%.

Key takeaway

For Machine Learning Engineers developing LLM applications that require accurate, up-to-date knowledge from Knowledge Graphs, consider adopting programmatic reasoning frameworks like Code-on-Graph. Your current reliance on predefined operators or direct factual injection into prompts may limit scalability and expressiveness for complex queries. Implementing schema-to-code generation can significantly improve performance, as CoG demonstrates up to a 10.5% gain on benchmarks, offering a more robust approach to mitigate hallucinations and outdated information.

Key insights

Code-on-Graph integrates LLMs with KGs using programmatic reasoning, representing schemas as Python classes for flexible, scalable knowledge utilization.

Principles

• Programmatic reasoning enhances LLM-KG integration.
• Abstract interfaces improve knowledge graph expressiveness.
• Avoid direct prompt injection for scalability.

Method

CoG identifies KG schemas, represents them as Python classes, generates executable code grounded in these classes, and instantiates retrieved facts as objects during execution.

In practice

• Develop Python class interfaces for KG schemas.
• Implement code generation for KG query execution.
• Test on complex question-answering datasets like WebQSP.

Topics

• Large Language Models
• Knowledge Graphs
• Programmatic Reasoning
• LLM-KG Integration
• Question Answering
• Python Classes

Best for: AI Engineer, Research Scientist, AI Scientist, Machine Learning Engineer, NLP Engineer

Related on AIssential

• Integrating Graphs, Large Language Models, and Agents: Reasoning and Retrieval — Integrating graphs with LLMs enhances reasoning, retrieval, and decision-making by combining structured knowledge with semantic understanding.
• Construction of Knowledge Graph based on Language Model — Lightweight LLMs can construct Hyper-Relational Knowledge Graphs with performance comparable to GPT3.5.
• DW-Bench: Benchmarking LLMs on Data Warehouse Graph Topology Reasoning — LLMs excel at orchestrating graph algorithms via tools but struggle with compositional multi-hop graph reasoning.
• Dynamic Large Concept Models: Latent Reasoning in an Adaptive Semantic Space — DLCMs improve LLM efficiency by shifting computation from tokens to a compressed, concept-based reasoning space.
• LEAP: Supercharging LLMs for Formal Mathematics with Agentic Frameworks — Agentic frameworks like LEAP enable LLMs to achieve state-of-the-art formal theorem proving by bridging informal reasoning with verifiable proofs.
• Why isn’t LLM reasoning done in vector space instead of natural language?[D] — LLMs internally use vectors, but natural language reasoning offers crucial interpretability and debuggability.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Artificial Intelligence.