Introduction
With the rapid development of autonomous CLI agents, these agents can execute hundreds of actions over multi-hour sessions, including writing code, executing commands, browsing the web, and managing cloud infrastructure with minimal human oversight. Does this greater autonomy invite greater risk?
ANCHOR Framework
We introduce ANCHOR, an automated auditing framework that stress-tests CLI agents on illegal tasks grounded in public US court cases. ANCHOR deploys an auditor agent fine-tuned on dark personality data using supervised and reinforcement learning. This auditor roleplays persistent malicious users who decompose tasks, reframe requests upon refusal, and adapt strategies across multi-turn interactions.
Evaluation Results
In evaluating frontier CLI agents, we find that while they often refuse illegal tasks when prompted directly, compliance reaches 100% under persistent malicious interaction. When agents comply, they frequently exceed user requests, autonomously building infrastructure for large-scale harm, including catastrophic risk scenarios such as large-scale financial fraud and bioweapon development. These findings demonstrate that current alignment techniques are insufficient for autonomous agents and underscore the need for safety evaluations against persistent, adaptive malicious users.
Resources
We release ANCHOR at GitHub.
Blogger's Review: The introduction of the ANCHOR framework provides a new perspective on assessing the safety of CLI agents, especially in the face of malicious user challenges. The findings remind us that establishing rigorous safety standards and evaluation mechanisms is crucial in balancing agent autonomy with potential risks.