Deep research and agent evolution serve as critical tasks for AI agents in real-world applications aimed at achieving artificial general intelligence. Deep research enables autonomous retrieval and integration of information in open-ended environments to tackle open-ended research tasks, yet it is limited by the static parametric capabilities of agent systems.
Conversely, agent evolution allows agents to autonomously interact with their environment to gain experiences that enhance model capabilities. However, its effectiveness has primarily been validated on verifiable tasks with standard answers, leaving a gap when it comes to open-ended research tasks.
To address this, we propose the Hybrid Open-Ended Tri-Evolution (HOTE) framework, which leverages hybrid-mode reinforcement learning to facilitate the collaborative evolution of a proposer, solver, and judge based on web-scale knowledge, moving towards autonomous evolving agents in open-ended tasks and environments.
Extensive experiments on three long-form deep research benchmarks demonstrate that the 8B model trained via HOTE outperforms the strongest static open 8-32B models as well as those trained by state-of-the-art deep research training methods, with reduced time overhead. Furthermore, the evolution of all three modules in HOTE is verified to be indispensable.
Blogger's Review: The HOTE framework presents a novel solution by integrating deep research and agent evolution, showcasing significant potential for autonomous learning in complex environments. This innovative approach not only optimizes model performance but also advances the AI field towards higher levels of achievement.