Real-time voice agents with Stream Vision Agents and Amazon Nova 2 Sonic

2026-05-14 · Source: Artificial Intelligence · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Software Development & Engineering, Cloud Computing & IT Infrastructure · Depth: Intermediate, long

Summary

This post details how to construct real-time voice agents using Stream's Vision Agents open-source framework, Amazon Bedrock, and Amazon Nova 2 Sonic. It addresses the engineering complexities of orchestrating speech-to-speech models, managing low-latency audio streaming, and handling connection lifecycles across various applications. The solution integrates Amazon Nova 2 Sonic, a speech-to-speech foundation model with real-time bidirectional audio streaming and function calling, with Stream's Vision Agents, a Python framework offering a plugin-based architecture and client SDKs. Stream's global edge network provides the real-time transport layer, ensuring sub-500ms join times and under 30ms audio latency. The architecture separates Stream's media transport from Amazon Nova Sonic's AI intelligence, which runs within the customer's AWS account, maintaining data control. The article provides code examples for setting up a basic agent and implementing function calling, highlighting the event-driven bidirectional streaming API of Nova 2 Sonic.

Key takeaway

For AI Engineers building conversational interfaces, leveraging Vision Agents with Amazon Nova 2 Sonic via Amazon Bedrock simplifies complex real-time voice agent development. You can rapidly deploy production-grade agents with features like function calling and multilingual support, significantly reducing infrastructure burden and focusing on core AI logic. Explore the provided code examples and documentation to implement custom functions and scale your voice applications.

Key insights

Combine Vision Agents with Amazon Nova 2 Sonic and Bedrock for production-ready, real-time voice agents.

Principles

• Abstract infrastructure complexity for AI experience customization.
• Separate media transport from AI intelligence for data control.
• Utilize bidirectional streaming for natural conversational flow.

Method

Integrate Vision Agents (Python framework) with Amazon Nova 2 Sonic (speech-to-speech model via Bedrock) and Stream's Edge Network for real-time media transport, managing audio flow and function calls.

In practice

• Use `uv add "vision-agents[getstream,aws]"` for setup.
• Define agent functions with `@llm.register_function` decorator.
• Employ `aws.LLM` for custom STT/TTS pipelines.

Topics

• Real-time Voice Agents
• Stream Vision Agents
• Amazon Nova 2 Sonic
• Amazon Bedrock
• Speech-to-Speech AI

Code references

• GetStream/Vision-Agents

Best for: AI Engineer, Machine Learning Engineer, MLOps Engineer

Related on AIssential

• Building real-time voice assistants with Amazon Nova Sonic compared to cascading architectures — Amazon Nova Sonic unifies speech processing for real-time, human-like voice AI, simplifying architecture and reducing latency.
• Scalable voice agent design with Amazon Nova Sonic: multi-agent, tools, and session segmentation — Multi-agent architectures and session segmentation are key to building scalable, low-latency voice agents.
• Issue #120 - Turn-based voice AI agents — Voice agents fall into turn-based (sequential) or streaming (overlapping) architectures, each with distinct trade-offs.
• Azure Speech at Build 2026: Powering Voice Agents with Real-Time and Life-like Experiences — Azure Speech updates enable rapid development of real-time, customizable, and production-ready voice agents.
• Build a Voice Agent in an Hour with Claude Code | AssemblyAI Workshop — The Voice Agent API simplifies voice AI development by integrating STT, LLM, and TTS with developer-friendly orchestration.
• OpenAI's NEW Voice Agent Model - GPT-RealTime 2 is dope! — OpenAI's new real-time voice models offer low-latency, bidirectional voice communication, translation, and transcription.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Artificial Intelligence.