Amazon SageMaker AI in 2025, a year in review part 2: Improved observability and enhanced features for SageMaker AI model customization and hosting

2026-02-20 · Source: Artificial Intelligence · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Cloud Computing & IT Infrastructure, Data Science & Analytics · Depth: Intermediate, medium

Summary

Amazon SageMaker AI introduced several enhancements in 2025 to improve training, tuning, and hosting of generative AI workloads. Key updates include enhanced observability features, such as granular instance-level and container-level metrics for CPU, memory, GPU utilization, and invocation performance, configurable via the `MetricsConfig` parameter in the `CreateEndpointConfig` API. Rolling updates for inference components now enable zero-downtime deployments with automatic rollbacks triggered by Amazon CloudWatch alarms, eliminating the need for duplicate infrastructure. Usability improvements feature serverless model customization, which automatically provisions compute resources for fine-tuning popular models like Amazon Nova and Llama, supporting techniques like RLVR and RLAIF. Additionally, bidirectional streaming facilitates real-time multi-modal applications by maintaining persistent connections, and expanded connectivity through AWS PrivateLink and IPv6 compatibility enhances security and compliance for enterprise deployments.

Key takeaway

For AI Engineers managing generative AI deployments, SageMaker AI's 2025 updates significantly streamline operations. You should explore enhanced metrics for granular performance insights and adopt rolling updates for safer, more efficient model deployments. Consider leveraging serverless model customization to accelerate fine-tuning processes and implement bidirectional streaming for real-time, conversational AI applications, ensuring compliance with PrivateLink and IPv6 support.

Key insights

SageMaker AI's 2025 updates enhance observability, deployment safety, and usability for generative AI workloads.

Principles

• Granular metrics improve diagnosis.
• Automated rollbacks enhance deployment safety.
• Serverless compute simplifies model customization.

Method

Enable enhanced metrics by adding `MetricsConfig` with `"EnableEnhancedMetrics": True` and `"MetricPublishFrequencyInSeconds"` to the `CreateEndpointConfig` API call for SageMaker endpoints.

In practice

• Use `MetricsConfig` for instance/container-level monitoring.
• Implement rolling updates for safer model deployments.
• Leverage serverless customization for fine-tuning models.

Topics

• Amazon SageMaker AI
• Generative AI Workloads
• Serverless Model Customization
• Real-time Inference
• Observability & Deployment

Best for: AI Engineer, NLP Engineer, Machine Learning Engineer, MLOps Engineer, AI Architect

Related on AIssential

• Amazon SageMaker AI in 2025, a year in review part 1: Flexible Training Plans and improvements to price performance for inference workloads — SageMaker AI's 2025 updates enhance generative AI inference with predictable capacity, improved resilience, and optimized performance.
• Migrate MLflow tracking servers to Amazon SageMaker AI with serverless MLflow — Migrating MLflow to SageMaker's serverless App reduces operational overhead and improves resource scaling.
• [AINews] The End of Finetuning — Finetuning is declining for general AI engineering but remains critical for top-tier applications and open models.
• Open Source Growth Boosts Together AI, Hugging Face — Open-source AI models and supporting services are rapidly gaining traction due to cost efficiency and vendor lock-in concerns, driving significant industry shifts.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Artificial Intelligence.