Why machines that can write poetry still can’t agree on what “good” means

2026-04-26 · Source: Naturallanguageprocessing on Medium · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Data Science & Analytics · Depth: Intermediate, medium

Summary

Agentic AI, despite its ability to generate answers and automate decisions, struggles with semantic ambiguity, where terms like "best" or "active" hold different meanings across various business contexts and departments. This "semantic gap" leads to AI agents providing confident but contextually incorrect answers, even with advanced prompt engineering or Retrieval-Augmented Generation (RAG). The article proposes that ontology-based semantic layers, particularly those built on SQL, are essential to ground AI agents in shared, governed meaning. These ontologies formally represent concepts, properties, and relationships, enabling AI to distinguish between context-dependent definitions (e.g., "active customer" for Sales vs. Finance) and operationalize this understanding through virtual knowledge graphs over existing data sources, thereby enhancing reliability and governance for enterprise-scale AI deployments.

Key takeaway

For CTOs and VPs of Engineering deploying agentic AI, relying solely on prompt engineering or RAG will lead to unreliable outcomes due to semantic ambiguity. You should prioritize implementing an ontology-based semantic layer, ideally one that is SQL-native, to provide a governed, explicit model of business meaning. This approach ensures your AI agents make contextually accurate decisions, reducing hallucinations and establishing a transparent chain of evidence critical for enterprise-grade accountability and compliance.

Key insights

Ontology-based semantic layers are crucial for grounding agentic AI in explicit, governed business meaning, overcoming inherent linguistic ambiguity.

Principles

• Fluency does not equal comprehension for LLMs.
• Semantic ambiguity is the default, not a bug.
• Shared meaning requires formal modeling.

Method

SQL ontologies create a virtual knowledge graph over existing data, mapping physical schemas to semantic concepts. This resolves ambiguity at the semantic layer, allowing agents to navigate context-specific definitions.

In practice

• Implement SQL ontologies for agentic AI.
• Define business terms explicitly in an ontology.
• Use LangChain/LangGraph with semantic layers.

Topics

• Agentic AI
• Semantic Gap
• Ontologies
• Knowledge Graphs
• SQL Ontologies

Best for: CTO, VP of Engineering/Data, Director of AI/ML, AI Engineer, MLOps Engineer, AI Architect

Related on AIssential

• Build Meaning Before Machines: Why Semantics, Ontologies, And Knowledge Graphs Matter For Agentic AI — Agentic AI requires explicit data meaning, making semantic layers and knowledge graphs essential architectural components.
• The Ontology Engineering Challenge is Not Building It, but Mirroring the Delta — The core challenge in enterprise AI is correcting LLM's inherent concept understanding where it diverges from specific business definitions.
• Google Just Entered the Semantic War. And It Brought a Different Weapon. — The core battle in enterprise AI is defining data meaning, not just model performance.
• Ontologies Are the Intentional Core of a True Knowledge Graph — Formal ontologies provide the stable, intentional semantic backbone essential for reliable enterprise AI and knowledge graphs.
• The Semantic Layer and AI Agents // David Jayatillake // MLOps Podcast #343 — A semantic layer standardizes data definitions, enabling AI agents to automate data tasks and answer business questions efficiently.
• The Semantics of Semantics — The data industry's "semantic layer" primarily offers SQL abstraction and metric governance, not formal semantic reasoning.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Naturallanguageprocessing on Medium.