OpenAI WebRTC Audio Session, now with document context

2026-06-12 · Source: Simon Willison's Weblog · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Software Development & Engineering · Depth: Intermediate, quick

Summary

Simon Willison has updated his "OpenAI WebRTC Audio Session" tool, initially developed in December 2024, to integrate OpenAI's new `GPT-Realtime-2` model. This model, introduced last month, is touted as OpenAI's "first voice model with GPT-5-class reasoning" and has a knowledge cutoff of September 30, 2024. A significant new feature allows users to paste document context into the web interface, enabling real-time audio conversations with the model about the provided information. This enhancement facilitates exploring specific documents conversationally through the browser, leveraging the advanced reasoning capabilities of `GPT-Realtime-2`, which is not yet available in the ChatGPT iPhone app. The tool offers a direct way to interact with OpenAI's latest real-time audio technology.

Key takeaway

For AI Engineers exploring real-time conversational interfaces, this updated tool offers a direct way to evaluate `GPT-Realtime-2`'s "GPT-5-class reasoning" with custom document context. You can prototype audio-driven document analysis or Q&A systems, leveraging the model's advanced capabilities before its broader availability. Consider using this playground to assess the model's performance on your specific data and use cases, informing future integration decisions.

Key insights

The "OpenAI WebRTC Audio Session" tool now enables real-time audio conversations with `GPT-Realtime-2` using user-provided document context.

Method

Users paste text into the "Document context" field, select `GPT-Realtime-2` and a voice, then start an audio session to converse with the model about the provided information.

In practice

• Explore documents conversationally via audio.
• Interact with `GPT-Realtime-2` before wider release.
• Test real-time audio models with custom data.

Topics

• OpenAI WebRTC API
• GPT-Realtime-2
• Real-time Audio
• Conversational AI
• Document Context
• AI Model Evaluation

Best for: Machine Learning Engineer, AI Product Manager, AI Engineer, NLP Engineer

Related on AIssential

• OpenAI’s New Voice Models Can Reason, Translate, and Transcribe – All While You’re Still Talking — OpenAI's new voice models enable real-time reasoning, translation, and transcription for highly interactive AI agents.
• OpenAI drops GPT 4.5 — GPT 5.4 introduces unprecedented steerability and multimodal capabilities with a 1 million token context window.
• OpenAI’s New Models Listen, Translate & Act in Real Time — OpenAI's new real-time audio models enable conversational AI agents to listen, translate, and act instantly.
• OpenAI launches new voice intelligence features in its API — OpenAI's new API models enable real-time voice interaction, translation, and transcription with advanced reasoning and safety.
• GPT 5.2: OpenAI Strikes Back — AI benchmark performance is increasingly driven by "thinking time" or test-time compute, complicating direct model comparisons.
• OpenAI releases GPT-5.5 Instant, a new default model for ChatGPT — GPT-5.5 Instant enhances performance and context management while reducing hallucinations in sensitive applications.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Simon Willison's Weblog.