Existing approaches for exploring cellular automata and other complex systems typically operate in an open‑loop fashion: they set initial conditions, run a full simulation, and passively observe the outcome without intervening during execution. This paper introduces a closed‑loop framework based on autotelic reinforcement learning, where an agent autonomously samples diverse goals and learns a goal‑conditioned policy to intervene in the system through minimal, local perturbations. We instantiate the framework on Lenia—a continuous cellular automaton known for life‑like self‑organizing patterns—and build an agentic system called CARL. Three core capabilities are demonstrated:
- Discovery of stable solitons: CARL discovers stable self‑organizing structures across a wide range of Lenia update rules at a rate significantly higher than heuristic baselines.
- Steering soliton motion: With only a few interventions, CARL learns to change the movement direction of existing solitons, showing that it can not only create but also control self‑organizing patterns.
- Human‑in‑the‑loop navigation: When humans provide high‑level directional commands, the trained agent translates them into low‑level perturbations in real time, guiding solitons through maze environments.
Trained over diverse goals, update rules, and random initial states, the agents acquire policies that zero‑shot generalize to various out‑of‑distribution conditions, demonstrating strong adaptability. These results suggest a viable path toward artificial experimentalist agents that, either autonomously or with human guidance, can discover and control emergent phenomena in complex systems, laying groundwork for future automated scientific discovery platforms.
Blogger's Review: This work elegantly merges autotelic reinforcement learning with a continuous cellular automaton, delivering a full closed‑loop pipeline from discovery to control. The real‑time human‑agent collaboration in maze navigation particularly highlights the framework’s practical potential. Extending this approach to higher‑dimensional or real‑world physical systems could open a new chapter for artificial experimentalists in scientific research.