Designing Data and AI Systems That Hold Up in Production

2026-02-26 · Source: Towards Data Science · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Data Science & Analytics, Software Development & Engineering · Depth: Advanced, medium

Summary

Mike Huls, a tech lead specializing in data engineering, AI, and architecture, emphasizes a full-stack perspective for building reliable data systems. He views data science models as integral parts of larger production systems, requiring robust data pipelines, APIs, and governance. Huls identifies recurring friction points across teams as signals for architectural or process-level issues worth addressing. He highlights that while AI agents are powerful, their complexity in production — involving state management, permissions, cost control, and failure handling — is often underestimated. He advocates for optimizing system architecture for change, even for small teams, by separating domain logic, application flow, and infrastructure concerns to enable evolution without constant rewrites. Huls also prioritizes correctness and traceability over raw speed in data insertion, especially for critical pipelines, and champions self-hosted, private AI solutions to ensure trust, auditability, and user control over data.

Key takeaway

For AI Architects designing and deploying agent-based systems, you should prioritize robust engineering practices and full-stack thinking from the outset. Underestimating the complexity of state management, cost control, and failure handling in production agents can lead to unpredictable, expensive, and risky operations. Focus on creating clear architectural boundaries and ensuring data integrity, as these foundational elements will become even more critical as generative AI matures into first-class production systems.

Key insights

Full-stack thinking and system-level design are crucial for building reliable, scalable, and trustworthy AI and data systems.

Principles

• Optimize architecture for change, not just initial delivery speed.
• Treat data science as a core part of a larger system.
• Correctness and traceability outweigh raw throughput for critical data.

Method

Identify structural problems by observing recurring team friction. Experiment with new technologies to understand trade-offs and solve real problems or reveal risks. Separate domain logic, application flow, and infrastructure concerns in architecture.

In practice

• Embed models in production systems with proper pipelines and APIs.
• Implement transactional safety for regulatory or financial data.
• Consider self-hosting AI for privacy, auditability, and cost control.

Topics

• AI Agents
• Data Architecture
• MLOps
• Data Privacy
• Generative AI

Best for: AI Architect, Data Scientist, AI Engineer, MLOps Engineer

Related on AIssential

• LAI #115: The Hidden Cost of “Agent-First” Thinking — AI system failures often stem from architectural and reproducibility issues, not just model performance.
• The Future of AI Is Stateful Infrastructure — Enterprise AI's future relies on stateful infrastructure managing context, not just model intelligence.
• Most AI Agents Fail in Production Because They’re Built Backwards — Production AI agents require a bottom-up, distributed systems architecture, not a top-down, model-centric approach.
• A Strong Suite in Agentic Strategy: Data Apps — AI agent success in enterprises depends on architecting governed, queryable data applications, not just advanced models or semantic layers.
• The Production AI Playbook: Deploying Agents at Enterprise Scale — Sandipan Bhaumik, Databricks — A structured five-pillar framework is essential for successfully deploying and managing enterprise-scale AI agents in production.
• Rebuilding the data stack for AI — Effective enterprise AI hinges on unified, governed, and accessible data infrastructure, moving beyond fragmented legacy systems.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Towards Data Science.