Identifying the Unknown: Prompt-Free Open Vocabulary Anomaly Recognition for Robot-Object Interaction

2026-06-25 · Source: Computer Vision and Pattern Recognition · Field: Technology & Digital — Robotics & Autonomous Systems, Artificial Intelligence & Machine Learning, Computer Vision · Depth: Expert, quick

Summary

AnomNOVIC is a novel two-stage framework designed for prompt-free open vocabulary anomaly recognition in robot-object interaction, addressing the need for robots to identify previously unseen objects in open-world environments. This known-workspace system integrates a masked autoencoder (MAE), which generates generic object-agnostic bounding boxes for anomaly detection, with NOVIC, a real-time, prompt-free open vocabulary image classifier. NOVIC then classifies these salient image regions without requiring a predefined candidate class list. Evaluated in a tabletop robot-object environment using the NICOL humanoid robot, AnomNOVIC achieved 47.1% AP and 57.5% AP50 for prompt-free recognition. When class candidates were provided, performance rose to 59.0% AP and 72.5% AP50. Across an additional in-the-wild test set featuring 48 unique objects, it reached up to 82.6% prompt-free detection and classification accuracy, significantly outperforming baselines like YOLO-World-v2, OWLv2, and YOLOE.

Key takeaway

For Robotics Engineers developing systems for open-world autonomy, AnomNOVIC offers a significant advancement in recognizing previously unseen objects without requiring explicit prompts. Your current object detection pipelines, especially those relying on predefined class lists or manual prompting, may be significantly less efficient and robust in dynamic environments. Consider evaluating AnomNOVIC's two-stage approach to enhance your robot's ability to interact with novel items and detect anomalies, potentially streamlining deployment in complex, unstructured settings.

Key insights

AnomNOVIC enables robots to recognize unknown objects efficiently and without prompts by combining a MAE with a real-time open vocabulary classifier.

Principles

• Open-world autonomy demands prompt-free object recognition.
• Two-stage detection can combine generic bounding boxes with classification.
• Masked autoencoders can generate object-agnostic regions.

Method

AnomNOVIC uses a MAE for generic object-agnostic bounding box generation, followed by NOVIC, a real-time prompt-free open vocabulary classifier, to identify salient image regions without predefined class lists.

In practice

• Deploy robots in dynamic, unknown environments.
• Enhance robotic manipulation of novel items.
• Improve anomaly detection in manufacturing.

Topics

• Open Vocabulary Recognition
• Anomaly Detection
• Robot-Object Interaction
• Masked Autoencoders
• Computer Vision
• Robotics

Best for: Research Scientist, Robotics Engineer, AI Scientist, Computer Vision Engineer

Related on AIssential

• Identifying the Unknown: Prompt-Free Open Vocabulary Anomaly Recognition for Robot-Object Interaction — AnomNOVIC enables prompt-free, open vocabulary object recognition for robots by combining an MAE for bounding boxes with a real-time classifier.
• Decoupled Object-Centric Video Understanding for Generating Robotic Manipulation Commands — Decoupling action recognition from object identification improves robotic command generation from video.
• Visual Prompting Meets Feature Reconstruction-Based Anomaly Detection with Dual-Teacher Supervision — Robust anomaly detection in varied real-world conditions is achieved by combining visual prompting, adaptive teacher models, and synthetic data.
• Detecting and Editing Visual Objects with Gemini — Gemini enables open-vocabulary object detection and advanced image editing using natural language prompts, eliminating traditional model training.
• MaskWAM: Unifying Mask Prompting and Prediction for World-Action Models — MaskWAM unifies mask prompting and prediction via a Mixture of Transformers to enhance robotic control by resolving spatial ambiguity and visual noise.
• 3D Instruction Ambiguity Detection — Detecting instruction ambiguity in 3D environments is critical for safe embodied AI, a gap current 3D LLMs fail to address.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Computer Vision and Pattern Recognition.