JANUS: Foreseeing Latent Risk for Long-Horizon Agent Safety

2025-12-25 · Source: cs.CL updates on arXiv.org · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Robotics & Autonomous Systems, Cybersecurity & Data Privacy · Depth: Expert, extended

Summary

JANUS introduces a foresight-oriented framework for long-horizon agent safety, addressing the challenge of preventing operational failures in tool-using agents before they act. The framework trains guard models, exemplified by Vanguard, to anticipate delayed risks from partial execution trajectories. It employs multi-agent simulation to synthesize diverse agent trajectories and utilizes a shared policy with two coupled tasks: an anticipation task that forecasts safety-relevant futures and an adjudication task that determines safety based on both observed context and anticipated future. These tasks are jointly optimized using CoAA-RL. Vanguard demonstrated significant improvements across four agent-safety benchmarks, increasing average protection by 15.9 percentage points over baseline guards and boosting benign task completion by 5.1 percentage points, achieving an average attack success rate (ASR) of 0.071.

Key takeaway

For AI Security Engineers or ML Engineers developing tool-using agents, adopting a predictive guarding framework like JANUS is crucial for mitigating long-horizon operational risks. Your current reactive safeguards are likely insufficient for complex agent workflows where harmful consequences manifest many steps after initial signals. You should explore integrating foresight capabilities, specifically training models to anticipate future unsafe trajectories, to proactively block actions and improve overall protection by over 15 percentage points without sacrificing benign task utility.

Key insights

Predictive guarding for tool-using agents significantly enhances safety by anticipating future risks from partial trajectories.

Principles

• Operational failures require foresight, not just reactive checks.
• Coupled anticipation and adjudication improves guard effectiveness.
• Simulated diverse trajectories enhance guard training.

Method

Janus trains a shared policy with an anticipation task (forecasting safety-relevant futures) and an adjudication task (deciding safety from observed and anticipated context), jointly optimized via CoAA-RL.

In practice

• Implement two-stage predictive judgment for agent actions.
• Categorize risks by user, environment, and agent origins.
• Use multi-agent simulation for diverse trajectory synthesis.

Topics

• Agent Safety
• Predictive Guardrails
• Reinforcement Learning
• Multi-agent Simulation
• Tool-using Agents
• Operational Risk

Code references

• xiongyuaay/JANUS

Best for: Research Scientist, CTO, VP of Engineering/Data, AI Scientist, Machine Learning Engineer, AI Security Engineer

Related on AIssential

• JANUS: Foreseeing Latent Risk for Long-Horizon Agent Safety — Agent safety can be significantly enhanced by foreseeing latent risks from partial trajectories before actions are executed.
• OSGuard: A Benchmark for Safety in Computer-Use Agents — OSGuard's dual-granularity benchmark reveals current guardrails excel locally but fail to ensure end-to-end safety in computer-use agents.
• Symbolic Guardrails for Domain-Specific Agents: Stronger Safety and Security Guarantees Without Sacrificing Utility — Symbolic guardrails offer provable safety and security for domain-specific AI agents without compromising utility.
• Frontier AI Risk Trends Are Splitting Apart: Misuse Safeguards Improve while Loss-of-control Safety Stagnates — Frontier AI risk trends are splitting, with misuse safeguards improving while loss-of-control safety stagnates despite capability increases.
• Operational Hallucination and Safety Drift in AI Agents — LLM agents exhibit Safety Drift and Operational Hallucination in multi-turn interactions due to reasoning-execution decoupling.
• HINTBench: Horizon-agent Intrinsic Non-attack Trajectory Benchmark — Intrinsic risks in long-horizon AI agents under benign conditions pose a significant, underexplored safety challenge.

See Counsel's argued verdicts on the open AI decisions leaders are weighing →

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by cs.CL updates on arXiv.org.