Equipping Large Language Model (LLM) agents with effective skills is crucial for solving complex tasks in real-world systems. This work aims to develop a framework that automatically constructs reusable skills, enhancing LLMs in tool use, multi-step reasoning, and dynamic environment interaction. We propose Collective Skill Tree Search (CSTS), a novel tree-search-based skill construction framework designed to build structured, diverse, and generalizable skill trees.
The core idea of CSTS is to leverage collective intelligence to jointly search, identify, and compose effective skills through two iterative phases: Collective Skill Node Generation (CSN-Gen) and Collective Skill Node Assessment (CSN-Assess). CSN-Gen exploits collective knowledge from multiple models to explore diverse candidate skills for each subtask, enabling comprehensive skill exploration. CSN-Assess employs multiple models as judges to evaluate and select skill nodes using two scoring mechanisms: (1) collective quality scoring, aggregating independent evaluations to produce a robust estimate of skill effectiveness; and (2) collective transferability scoring, explicitly verifying whether a skill generalizes well across different models.
With CSTS, we construct a comprehensive tree of skills along with skill-augmented training data, enabling models to effectively learn and utilize skills. Additionally, we introduce Collective Skill Reinforcement Learning, which actively selects multiple relevant skills from the tree to broaden solution-space exploration, avoiding being trapped by a single skill and its resulting homogeneous or suboptimal solutions. Consequently, our trained model, OpenClaw-Skill, exhibits outstanding agentic capabilities in long-horizon planning, tool use, and generalization over challenging benchmarks.
Blogger's Review: The CSTS framework proposed in this paper builds skill trees using collective intelligence, significantly enhancing LLM performance in complex tasks. The multi-model evaluation mechanism improves both skill effectiveness and transferability, making it noteworthy. Future applications may change how LLMs adapt in dynamic environments.