EmbodiedCLBench is the first benchmark for continual learning in embodied simulation, designed to assess the learning capability of self‑evolving agents. The core evaluation principle is compositional generalization: agents first learn basic tasks and are then tested on whether they can recombine acquired skills to solve unseen advanced tasks. Using this benchmark we conducted extensive experiments across multiple agent frameworks and learning methods, revealing consistent limitations and distinctive behaviors. Results show that advanced tasks are almost unsolvable in a zero‑shot setting, yet experience from basic tasks yields stable improvement even as task complexity grows. However, the improvement does not scale linearly with the number of training samples, and current agents fail to extract meaningful information from additional experience. Overall, the findings highlight continual learning as a fundamental yet under‑explored capability, and the benchmark provides valuable resources for designing effective self‑evolving embodied agents. Code and dataset are publicly available.
Review: This benchmark bridges a critical gap in embodied continual learning evaluation, offering a clear experimental foundation for future algorithmic advances.