Clicky Meets Universal-3.5: Voice-Navigate Your Code

2026-06-30 · Source: AssemblyAI · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Software Development & Engineering · Depth: Intermediate, quick

Summary

AssemblyAI has enhanced its voice-navigation tool, Clicky, by integrating the newly launched Universal 3.5 transcription model. Previously running on Universal 3 Pro Streaming, Clicky now utilizes Universal 3.5, AssemblyAI's most accurate transcription model, to significantly improve its understanding of technical code terms. The system operates by taking a screenshot of the current page, then dynamically updating the transcription model's "key terms" with on-screen text. This proactive anticipation allows Universal 3.5 to accurately transcribe and interpret complex or dictionary-absent terms like "PCMS16LE" and "audio PCM 16 data," which are crucial for navigating code. Demonstrations showed Clicky identifying "PCMS16LE" on line 464 and "audio PCM 16 data" on line 191, illustrating the model's ability to precisely locate and explain code elements based on voice commands. A "Key Terms Widget" further visualizes this dynamic updating process.

Key takeaway

For NLP Engineers building voice-controlled applications, especially those involving technical or domain-specific language, you should consider integrating visual context to dynamically prime your ASR models. AssemblyAI's Universal 3.5 demonstrates that feeding on-screen text as anticipated key terms significantly boosts transcription accuracy for complex vocabulary. This approach can enhance user experience by enabling precise voice navigation and command execution in specialized interfaces like code editors.

Key insights

Dynamic key term updating from screen content significantly enhances speech-to-text accuracy for domain-specific vocabulary, enabling precise voice navigation.

Principles

• Contextual screen data enhances transcription.
• Dynamic key term anticipation improves accuracy.
• Visual input guides audio processing.

Method

Clicky captures screen content, extracts text, and dynamically updates AssemblyAI's Universal 3.5 transcription model's anticipated key terms. This contextual priming enables accurate voice command processing for code navigation.

In practice

• Prime ASR models with on-screen text.
• Dynamically update transcription dictionaries.
• Combine visual context with voice commands.

Topics

• Voice Navigation
• Speech-to-Text
• AssemblyAI Universal 3.5
• Contextual ASR
• Code Editors
• Dynamic Key Terms

Best for: AI Engineer, NLP Engineer, Prompt Engineer

Related on AIssential

• Live demo of AssemblyAI's Universal-3.5 Pro Realtime Speech-to-Text model — Contextual prompting and noise suppression significantly enhance real-time speech-to-text accuracy across diverse languages and challenging audio environments.
• Code Switching in Real-Time | Universal-Streaming Speech-to-Text — Assembly AI's universal model enables real-time, zero-latency code-switched speech-to-text across six languages.
• Voice of India: A Large-Scale Benchmark for Real-World Speech Recognition in India — Real-world Indic ASR requires benchmarks with unscripted speech, diverse languages, and flexible spelling.
• AST: Adaptive, Seamless, and Training-Free Precise Speech Editing — AST enables precise, training-free speech editing by recomposing latent representations from a pre-trained TTS model.
• From Text to Voice: A Reproducible and Verifiable Framework for Evaluating Tool Calling LLM Agents — Converting text-based tool-calling benchmarks to audio reveals significant model- and task-dependent performance gaps.
• Pretrained self-supervised speech models can recognize unseen consonants — Pretrained self-supervised speech models effectively generalize to and accurately recognize typologically rare phonemes, such as click consonants.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by AssemblyAI.