Oppo open-sources Android AI agent X-OmniClaw that uses your camera, screen, and voice without leaving the phone

2026-05-17 · Source: The Decoder · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Robotics & Autonomous Systems · Depth: Intermediate, medium

Summary

Oppo's Multi-X team has open-sourced X-OmniClaw, an Android AI agent that operates directly on physical devices using the phone's camera, screen, and voice. Unlike cloud-based solutions, X-OmniClaw processes data locally, with a cloud language model only providing "fuel" for complex reasoning. The system integrates multiple perception channels, processes gallery photos into a searchable text-based memory during idle times, and learns user behavior to clone actions via deeplinks instead of replaying tap paths. Demos showcased its ability to compare product prices using the camera, act as a "ScreenAvatar" for on-screen tasks like solving exercises, and autonomously create photo albums from a user's gallery. The project builds on the HermesApp codebase and is available on GitHub.

Key takeaway

For CTOs or VPs of Engineering evaluating on-device AI solutions, X-OmniClaw demonstrates a viable architecture for privacy-preserving, multi-modal mobile agents. Your teams should investigate its open-source codebase to understand how local perception, memory, and action cloning can enable advanced user experiences without relying on cloud-based virtualization or compromising sensitive user data.

Key insights

X-OmniClaw is an on-device Android AI agent integrating multiple perception channels for local task execution.

Principles

• Prioritize on-device processing for privacy and sensor access.
• Combine multiple perception channels for robust understanding.
• Clone user behavior for efficient, reusable task execution.

Method

The agent combines vision-language interpretation with camera, screen, and voice inputs, processes gallery photos into a searchable Markdown memory, and clones user tap paths into deeplink-based skills.

In practice

• Use on-device agents for sensitive data tasks.
• Implement multi-modal perception for richer context.
• Employ behavior cloning for app automation.

Topics

• X-OmniClaw
• Android AI Agent
• On-device AI
• Multi-modal Perception
• User Behavior Cloning

Code references

• OPPO-Mente-Lab/X-OmniClaw

Best for: CTO, VP of Engineering/Data, Director of AI/ML, AI Engineer, Machine Learning Engineer, AI Scientist

Related on AIssential

• Open source for awkward robots — Open-source software and immutable blockchain-based guardrails are critical for trustworthy, adaptable humanoid robotics.
• iOSWorld: A Benchmark for Personally Intelligent Phone Agents — Personally intelligent phone agents require benchmarks that simulate persistent user identity and complex, multi-app interactions.
• Automate Your Life with PicoClaw AI Assistant and Raspberry Pi Zero 2W — PicoClaw on Raspberry Pi Zero 2W enables ultra-low-power, continuous AI automation on edge devices.
• My Most INSANE AI Agent Ever (OpenClaw Clone) — A simplified AI agent system automates complex tasks, including video creation, by integrating browser-based interactions and scheduled jobs.
• OpenMobile: Building Open Mobile Agents with Task and Trajectory Synthesis — OpenMobile provides an open-source framework for synthesizing mobile agent training data, improving performance and transparency.
• OpenClaw That Runs on $10 Hardware — Efficient AI agents can run on minimal hardware and integrate deeply with web platforms and development workflows.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by The Decoder.