The Data Agent Stack - Part 4: Meaning Lives in Code

2026-02-17 · Source: The Agent Stack · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Data Science & Analytics, Software Development & Engineering · Depth: Advanced, long

Summary

Data agents often fail to provide accurate answers despite correct SQL due to a lack of understanding of data's true meaning, which extends beyond mere schema definitions. While schemas describe data shape and query history shows usage patterns, neither fully captures critical context like data grain, exclusions, freshness, or business rules. The article argues that pipeline code is the highest-authority source for this "hidden contract," revealing executable behavior such as deduplication rules, update cadences, and implemented business logic. It proposes building a comprehensive "table context card" that integrates schema metadata, trusted query usage, owner annotations, semantic definitions, lineage, orchestration state, and pipeline-derived facts. This approach, exemplified by systems like OpenAI's in-house data agent and Spotify's data assistant, ensures agents reason over governed, reliable context rather than anonymous or potentially misleading information.

Key takeaway

For Data Engineers or AI Engineers building data agents, relying solely on table schemas or query history will lead to inaccurate results. You must integrate pipeline code as a high-authority source to capture the true meaning, grain, and freshness of your data. Implement a robust "table context card" system that extracts and refreshes code-derived facts, ensuring your agent reasons over governed, validated context. This approach prevents common failure modes like grain mismatches or stale logic, making your data agent reliable and trustworthy.

Key insights

Data agents require code-derived context to understand true data meaning, beyond schemas and query history, for reliable answers.

Principles

• Pipeline code reveals executable data behavior and hidden contracts.
• Data meaning encompasses grain, exclusions, freshness, and business rules.
• Context sources must be ranked by authority and reconciled.

Method

Build a "table context card" by extracting and refreshing table-centered facts from pipeline code, integrating schema, query history, annotations, lineage, and orchestration state.

In practice

• Create a table context card schema with key attributes.
• Extract executable behavior from dbt models, Spark jobs, SQL files.
• Validate context changes with golden questions.

Topics

• Data Agents
• Pipeline Code
• Data Context
• Data Governance
• Data Lineage
• Semantic Layer

Best for: AI Engineer, Data Engineer, Data Scientist

Related on AIssential

• The Data Agent Stack - Part 1: What Is a Data Agent? — A data agent is a governed analysis loop that proves answers, not just generates SQL, by integrating context and validation.
• Building Agent Ready Data Pipelines From Scratch — AI agents require data pipelines to provide rich metadata for accurate interpretation, beyond traditional data quality.
• The Data Agent Stack - Part 2: The Data Foundation Is the Agent — The reliability of data agents hinges on a well-governed data foundation that explicitly encodes meaning, quality, and ownership for machine consumption.
• Agentic Data Engineering with Genie Code and Lakeflow — Genie Code enables natural language generation and orchestration of production-ready data pipelines, accelerating data engineering workflows.
• Why Enterprise AI Needs a Governed Meaning Layer: Introducing Snowflake Horizon Context — Enterprise AI requires a governed meaning layer like Snowflake Horizon Context to resolve semantic ambiguity and prevent hallucinations.
• Context intelligence for your data and AI agents at scale — AWS innovations provide scalable, governed context intelligence for AI agents by unifying scattered data into knowledge graphs.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by The Agent Stack.