Abstract
Agent safety is shifting focus from content moderation to preventing operational failures before tool-using agents act. We propose Janus, a foresight-oriented framework for long-horizon agent safety that trains guards to anticipate delayed risks from partial trajectories.
Janus synthesizes diverse agent trajectories via multi-agent simulation and learns a shared policy with two coupled tasks: an anticipation task that forecasts safety-relevant futures and an adjudication task that decides safety from both the observed prefix and anticipated future. These two tasks are jointly optimized with CoAA-RL, which rewards forecasts based on their utility for downstream safety judgment.
The resulting guard model, Vanguard, blocks unsafe actions before execution. Across four agent-safety benchmarks, Vanguard improves average protection by 15.9 percentage points over baseline guards while increasing benign task completion by 5.1 percentage points.
Blogger's Review: The Janus framework effectively enhances agent safety, particularly in long-horizon scenarios, showcasing significant practical value. The success of the Vanguard model highlights the potential of foresight prediction in reinforcement learning, warranting further exploration and application.