Intelligence is Free, Now What? Data Systems for, of, and by Agents

2026-07-07 · Source: The Berkeley Artificial Intelligence Research Blog · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Data Science & Analytics · Depth: Advanced, long

Summary

The rapid decline in AI inference costs, from approximately \$30 per million GPT-4-class tokens in early 2023 to under \$1 today, with some providers pushing below \$0.10, is ushering in an era of "virtually free intelligence." This shift, marked by a median 50x annual price drop across benchmarks, presents three new challenges and opportunities for data systems. First, "Data Systems For Agents" addresses redesigning systems for agentic speculation, where agents generate high-volume, heterogeneous queries, requiring optimizations like multi-query reuse and approximate answers. Second, "Data Systems Of Agents" focuses on the agentic substrate, including structured memory for efficient retrieval and mechanisms for concurrent edits, coordination, and failure handling in multi-agent swarms. Third, "Data Systems By Agents" explores agents synthesizing custom, disposable data systems from scratch for specific workloads, with challenges in verification and ensuring trustworthiness.

Key takeaway

For AI Architects and Data Engineers designing future systems, the advent of near-free intelligence means you must fundamentally rethink data system paradigms. Your designs should anticipate agentic workloads by incorporating multi-query optimization and approximate query processing. You should also prioritize robust structured memory and coordination mechanisms for multi-agent swarms, and explore how agents can synthesize custom, verifiable data systems for specific needs.

Key insights

Near-free AI intelligence necessitates a fundamental re-architecture of data systems for, of, and by agents.

Principles

• Agentic speculation requires data system redesign.
• Multi-agent swarms need robust state management.
• Agents can synthesize custom, disposable data systems.

Method

Agents can synthesize workload-specific analytical engines or key-value stores by generating code from specifications, then using auxiliary verification agents to generate test cases and expand specifications.

In practice

• Implement multi-query optimization for agent workloads.
• Design structured memory for agent knowledge.
• Explore agent-driven custom system synthesis.

Topics

• AI Agents
• Data Systems Architecture
• Inference Cost Reduction
• Agentic Speculation
• Structured Memory
• Custom System Synthesis
• Multi-Agent Systems

Code references

• vllm-project/vllm
• sgl-project/sglang
• openai/codex
• skydiscover-ai/skydiscover

Best for: Research Scientist, CTO, VP of Engineering/Data, AI Scientist, Data Engineer, AI Architect

Related on AIssential

• I Thought One Prompt Could Do Everything… I Was Wrong — Multi-agent AI systems, mimicking human teamwork, overcome single-model "context overload" by distributing tasks.
• Data Intelligence Agents: Interpreting, Modeling, and Querying Enterprise Data via Autonomous Coding Agents — Autonomous coding agents can effectively interpret, model, and query enterprise data by generating and executing artifacts.
• OpenAI's AI data agent, built by two engineers, now serves 4,000 employees — and the company says anyone can replicate it — OpenAI's internal AI data agent demonstrates the transformative power of accessible data intelligence for enterprise operations.
• Before You Build an AI Agent, Read This — Effective AI agent development prioritizes foundational workflows and adaptive retrieval over static, single-tool approaches.
• not much happened today — The AI industry is rapidly integrating hardware, software, and agentic systems, intensifying competition and raising new infrastructure and security challenges.
• Building AI Agent Systems and Scaling Challenges in Agentic AI — Scaling AI agents requires architectural decomposition into multi-agent systems to manage complexity and prevent failure propagation.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by The Berkeley Artificial Intelligence Research Blog.