AI systems are increasingly engaging in their own improvement: revising outputs, adapting during deployment, training on generated data, and even conducting AI research autonomously. The literature uses terms like "self-refine," "self-reward," "self-play," and "self-evolve," which conflate fundamentally different objectives.
We surveyed 1,250 arXiv papers (2024-2026) along two axes: what the system improves—its behavior in deployment, its policy through training, its evaluator, or the research process itself—and the degree of loop closure (from human-in-the-loop to fully closed).
The taxonomy distinguishes bounded self-refinement—convergent, evaluable, and already industrial practice—from open-ended recursive self-improvement (RSI), which remains constrained by grounding requirements, collapse dynamics, and compute limitations. A unique aspect is the dedicated category for self-evaluation: each improvement loop claims that some signal can substitute for human judgment.
We explored the evaluator design space—judges, process reward models, verifiers, rubrics, meta-evaluation—and ordered the signals into a verification hierarchy from formal verifiers (strongest) to intrinsic self-assessment (weakest). Demonstrated self-improvement strength tracks this hierarchy, with failure modes (self-confirming loops, model collapse, diversity collapse) arising from violations, and the "research direction-setting" bottleneck maintaining human involvement sits at the top of this hierarchy.
We connect the technical literature to RSI limits theory and address safety and governance questions raised by frontier-lab accounts of closing the loop, identifying governance-grade measurement of self-improvement as the most underpopulated niche in the field.
Blogger's Review: This article provides a profound insight into the complexities of AI self-improvement, emphasizing the importance of distinguishing between bounded and open-ended self-improvement. As AI's self-optimization capabilities grow, ensuring their safety and effectiveness will be crucial for future research.