How OpenAI Built Its Data Agent

2026-07-22 · Source: ByteByteGo · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Data Science & Analytics, Robotics & Autonomous Systems · Depth: Intermediate, short

Summary

OpenAI's internal data agent, serving thousands of employees daily, employs a surprisingly simple architecture to answer natural language queries by generating and executing SQL. This agent, powered by GPT 5.5 and 13 tools, relies heavily on a "context assembly layer" to navigate over 70,000 database tables. This layer creates comprehensive table descriptions from usage metadata, human annotations, and nightly Codex enrichment, alongside company knowledge and past agent corrections. Beyond context assembly, OpenAI leverages Codex internally for critical tasks like migrating 90,000 tables and 600 petabytes in two months, managing patches for a dozen open-source forks, and assisting support engineers with around 100 fixes daily. Key engineering lessons include prioritizing data foundations, limiting tool complexity, trusting high-quality past queries, guiding goals over specific steps, and setting ambitious timelines.

Key takeaway

For AI Engineers developing internal data agents, prioritize establishing a robust data foundation and a sophisticated context assembly layer over complex multi-agent architectures. Your agent's reliability hinges on high-quality table descriptions and curated knowledge, not an abundance of tools or fine-tuning. Consider leveraging large language models like Codex for automating data migrations, patch releases, and support tasks to achieve ambitious operational timelines.

Key insights

OpenAI's data agent achieves reliability through architectural simplicity and robust context assembly, not complex multi-agent pipelines.

Principles

• Data foundation quality is paramount for agent reliability.
• Fewer, focused tools enhance agent performance.
• Guide agent goals, not detailed execution paths.

Method

The agent's core method involves a context assembly layer that retrieves relevant table descriptions (from metadata, human notes, Codex enrichment), company knowledge, and memory to build a comprehensive prompt for the LLM.

In practice

• Use Codex for data migration and patch management.
• Implement trust-based ranking for historical queries.
• Annotate tables with human notes and usage metadata.

Topics

• OpenAI Data Agent
• LLM Agents
• Context Assembly
• Data Platform Engineering
• Codex Automation
• SQL Generation

Best for: AI Architect, CTO, VP of Engineering/Data, AI Engineer, Machine Learning Engineer, MLOps Engineer

Related on AIssential

• How OpenAI Built Its Data Agent — The reliability of AI agents at scale stems from strong data foundations and carefully engineered context, not complex agent architectures.
• OpenAI's AI data agent, built by two engineers, now serves 4,000 employees — and the company says anyone can replicate it — OpenAI's internal AI data agent demonstrates the transformative power of accessible data intelligence for enterprise operations.
• LAI #117: Why Reliable AI Systems Are Still So Hard to Build — AI agent development requires robust engineering for reliability, observability, and controlled autonomy to move beyond hype.
• Use subagents and custom agents in Codex — OpenAI Codex now offers subagents and custom agents for modular, specialized AI-driven development tasks.
• Presentation: AI Agents to Make Sense of Data at OpenAI — OpenAI's Kepler agent uses advanced AI techniques to automate complex data analysis across vast datasets, improving data accessibility and accuracy.
• Everyone Wants an AI Agent. Nobody Wants to Talk About the Data. — AI agent effectiveness is bottlenecked by data quality and infrastructure, not just model intelligence.

See Counsel's argued verdicts on the open AI decisions leaders are weighing →

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by ByteByteGo.