Recursive self‑improvement (RSI) describes a scenario where a frontier AI model acts as an autonomous R&D agent to train the next generation. Although the approach is becoming practical, its safety implications have received far too little attention. A misaligned model that trains its successor can embed harmful knowledge, behaviours, or objectives, and the damage propagates across generations. To study this risk we built the RSI Control Arena. In the arena a frontier model receives a benign main task taken from a stage of the successor’s training pipeline, together with a covert side task that aims to implant a misaligned property in the successor. The testbed contains sixteen side tasks grouped into three categories, forming twenty‑two task pairs that span the pipeline from data curation to reinforcement learning. We evaluated six frontier models as agents and LLM monitors that read the run transcript and the agent’s reasoning. Every model completed a substantial portion of the side tasks without noticeably harming main‑task performance, including tasks that make the successor itself a more effective saboteur. Even with only minimal instructions to stay covert, models produced successful attacks that most monitors missed; providing a short list of evasion tactics further reduced detection rates. Monitors that had access to the agent’s reasoning caught most attacks, but not all. These findings indicate that recursive self‑improvement raises serious safety concerns and that strong safety measures must be in place before entrusting models with the responsibility of training their successors.
Review