A central challenge in embodied recursive self‑improvement is enabling agents to turn interaction histories into capabilities that persist across worlds and transfer to later tasks. Simply storing a memory or achieving a final action does not prove a reusable capability has been acquired; evaluation must separate true capability change from carried‑over resources, unequal source exposure, and early execution failures. To address this we built MC‑EvoBench, a controlled Minecraft benchmark containing 19 registered evolution chains—12 primary evaluation chains, seven calibration or probe chains, and three graph‑aligned paired probe families for transfer testing. Each task is represented as a task‑state graph, with stage‑level environment verifiers, matched source‑target roles, a reset of the target world, and only a declared agent artifact allowed to cross the source‑target boundary. For every probe we evaluate no‑prior, related, unrelated, and shuffled source conditions under identical environment‑step budgets and five random seeds, reporting TSR, WGP, and VMP while retaining provenance records. Results reveal a consistent direct‑capability gap between controller and planner‑backed variants; related experience yields no stable transfer—it does not improve tool probes, its modest wood‑species gain appears in only one seed, and all confidence intervals include zero. Stage audits further show that early wood‑collection and crafting bottlenecks often prevent later capabilities from being observed. MC‑EvoBench thus offers a controlled testbed for assessing durable, transferable embodied capabilities. Code and data are open‑source at https://github.com/xzw-ustc/mc-bench-open-source.
Review