Agent systems are moving from single‑task tool agents to persistent, personalized claw‑style assistants such as OpenClaw. These agents keep configuration files (e.g., AGENTS.md), long‑term memory, and reusable skills; once poisoned, the persistent state is reloaded each time, creating long‑lasting threats across tasks and sessions. Existing security evaluation frameworks rely on manually crafted test cases and environments, which cannot fully cover the complexity of claw‑style systems, become outdated as agents evolve, and mismatch real deployment scenarios.
To address this gap we introduce AutoRedClaw, an automated red‑team framework that generates end‑to‑end attacks together with the environments needed to run them. The framework enforces a strict quality‑checking protocol and execution‑based validation to guarantee correctness, realism, and effectiveness of the samples, while task‑driven environment construction keeps costs extremely low. For broad coverage, AutoRedClaw organizes generation by risk categories, attack strategies, attack stages, and attack surfaces, and uses experience‑guided exploration to steer generation toward under‑explored regions.
Experiments show that AutoRedClaw achieves an attack success rate above 30% at a cost of less than $0.10 per successful sample. From its outputs we build RedClaw‑Bench, a benchmark of 330 cases whose attack success rates significantly surpass those of AgentDojo, AgentDyn, and DTap. Across seven harnesses and four frontier models, most agent systems exceed a 50% success rate, demonstrating that the red‑team pipeline provides a strong security signal even for the most advanced agents. The code and benchmark are released at https://github.com/SaFo-Lab/AutoRedClaw.
Review: AutoRedClaw fills a critical gap in the security assessment of claw‑style agents by automatically producing high‑quality attack samples and corresponding environments, offering researchers a scalable, low‑cost, and trustworthy red‑team testing tool.