Monitoring & Observability in Microsoft Foundry

2026-05-22 · Source: Microsoft Foundry Blog articles · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Cloud Computing & IT Infrastructure, Cybersecurity & Data Privacy · Depth: Intermediate, long

Summary

Microsoft Foundry now offers generally available (GA) Evaluation, Monitoring, and Tracing capabilities through its Control Plane, deeply integrated with Azure Monitor since March 2026. This system addresses the unique challenges of production AI, where agent behavior can shift without code changes, unlike traditional applications. Foundry's three pillars include continuous evaluation with built-in and custom evaluators for critical quality and safety dimensions (e.g., Coherence, Groundedness, Retrieval Quality), integrated monitoring publishing all observability data to Azure Monitor for cross-stack correlation and unified alerting, and OpenTelemetry-based distributed tracing to pinpoint failure causes. The platform also centralizes security, compliance, and cost governance, leveraging Defender for Cloud and Microsoft Purview to mitigate risks like prompt injection and data leakage, and providing detailed token usage and quota monitoring.

Key takeaway

For MLOps Engineers and AI Architects deploying production agents, Microsoft Foundry's GA observability features fundamentally change how you ensure AI quality and safety. You should integrate Foundry's continuous evaluation, monitoring, and tracing with your existing Azure Monitor setup to gain unified visibility and proactive alerting. This allows you to quickly diagnose issues like groundedness drops or security risks, moving beyond episodic testing to continuous operational governance and cost optimization.

Key insights

Production AI requires continuous, integrated observability beyond traditional APM to manage dynamic behavior and ensure quality, safety, and cost.

Principles

• AI evaluation must be continuous, not episodic.
• Unify observability data for cross-stack correlation.
• Link evaluation results directly to traces for root cause.

Method

Foundry's approach combines continuous evaluation (built-in/custom evaluators), integrated monitoring (Azure Monitor), and OpenTelemetry-based tracing, all managed via Foundry Control Plane for unified oversight.

In practice

• Configure 5-10% sampling for production queries.
• Use LLM-as-a-Judge for nuanced quality checks.
• Enable Defender for Cloud for AI threat protection.

Topics

• Microsoft Foundry
• AI Observability
• Continuous Evaluation
• Azure Monitor
• Distributed Tracing
• AI Governance

Best for: CTO, VP of Engineering/Data, Director of AI/ML, MLOps Engineer, AI Architect, DevOps Engineer

Related on AIssential

• Generally Available: Evaluations, Monitoring, and Tracing in Microsoft Foundry — Continuous evaluation and integrated observability are crucial for maintaining AI agent quality in dynamic production environments.
• AI Observability Starter Kit for Microsoft Foundry agents — Production-grade AI observability requires instrumented traces, automated quality and safety evaluations, and proactive alerting.
• Mind the Gap (In your Agent Observability) — Amy Boyd & Nitya Narasimhan, Microsoft — Effective AI agent observability requires continuous evaluation, monitoring, and automated optimization to manage non-determinism and ensure reliability.
• The Next AI Observability Problem Is Semantic, Not Just Operational — AI observability must shift from operational health to semantic understanding of decision quality within workflows.
• Building Observability for a Production GenAI System: An Internal Knowledge Base End-to-End — Integrated observability across cost, quality, and latency is critical for production GenAI systems.
• Why enterprise AI ROI starts with observability — AI observability links model behavior to business outcomes, enabling precise ROI measurement and risk mitigation.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Microsoft Foundry Blog articles.