Why I Refused to Expose Pydantic AI to My Frontend — Research Notes

2026-07-09 · Source: Towards AI - Medium · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Software Development & Engineering · Depth: Intermediate, long

Summary

ScriptorDB implemented an application-level event translation layer to address the challenge of exposing Pydantic AI's internal event stream to its production frontend. While Pydantic AI's agent.run() provides only a final result, a web application requires real-time visibility into the execution process, including tool calls and their outcomes. The team deliberately avoided exposing Pydantic AI's internal ModelMessage structures, which are unstable and overly complex for frontend needs. Instead, they created a stable set of six application-level events: run_start, text_delta, tool_call, tool_result, trace, and run_end. This architecture, comprising agent_runner.py for translation, streaming.py for SSE protocol adaptation, and chat.py for HTTP lifecycle and persistence, ensures the frontend uses a simple reducer for state management. This design, which limits Pydantic AI to 73 lines of codebase, offers protocol agnosticism, enhanced testability, and improved observability.

Key takeaway

For AI Architects designing production LLM applications with real-time user interfaces, you should implement an application-level event translation layer. This prevents coupling your frontend and persistence to volatile LLM framework internals, ensuring system stability and testability across updates. Define a stable, application-specific event contract to simplify frontend state management and enable protocol agnosticism. This approach enhances observability and reduces future refactoring efforts.

Key insights

Decouple LLM framework internals from frontend and persistence layers using an application-level event translation.

Principles

• Never expose LLM framework internals.
• Decouple layers by single responsibility.
• Frontend needs process, not just result.

Method

Translate raw framework events into stable application events (e.g., run_start, tool_call, tool_result). Wrap these into SSE strings for the frontend, handling HTTP lifecycle and persistence separately.

In practice

• Implement a dedicated event translation layer.
• Use asyncio.Queue to decouple event producer/consumer.
• Filter internal messages before sending to frontend.

Topics

• Pydantic AI
• LLM Application Architecture
• Event Streaming
• Server-Sent Events
• Decoupling Architecture
• Agent Orchestration

Best for: AI Engineer, MLOps Engineer, AI Architect

Related on AIssential

• Unified Agentic Memory Across Harnesses Using Hooks — A shared, external memory layer using standardized hooks prevents LLM agent vendor lock-in.
• Issue #128 - Structured LLM Outputs with Pydantic — Combine Pydantic schemas with LangChain's output parsers and LCEL for robust, structured LLM outputs.
• The 3 Invisible Risks Every LLM App Faces (And How to Guard Against Them) — LLM applications introduce unique security risks requiring specialized guardrails beyond traditional software security.
• Building an Agentic AI App With a Local LLM — No Cloud, No API Costs — Local LLMs on consumer GPUs enable cost-free, private AI app development with cloud-ready architecture.
• [AINews] Everything is Conductor — The AI ecosystem is rapidly converging on agent-first development, driving innovation in tooling, infrastructure, and autonomous systems.
• Most AI Agents Fail Because They Are Built Like Chatbots — AI agents need robust, external state management, not chat history, for reliable enterprise workflows.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Towards AI - Medium.