SkillCT is a multidimensional benchmark for assessing agent skills, designed to go beyond traditional evaluations that focus solely on downstream task execution. It defines nine complementary dimensions of skill quality, grouped into three aspects: Static Assessment (capturing intrinsic properties of the skill specification), Dynamic Assessment (measuring behavior stability across different backbone models and request variations), and Human‑Assisted Assessment (examining consistency of the behavior chain and task granularity). These dimensions provide a broader view of quality than mere execution success.
Applying SkillCT to a public skill corpus yields a reference benchmark that reveals quality weaknesses across dimensions and their inter‑relationships. Building on this benchmark, we introduce an execution‑free quality evaluation baseline: by retrieving and leveraging similar reference skills, the multidimensional quality profile of unseen skills can be predicted without actually running them. This approach dramatically reduces token consumption and evaluation time while preserving consistency with the full assessment.
Further experiments show that SkillCT‑guided optimization improves downstream task success rates and execution efficiency, demonstrating its practical value for skill refinement, scalable assessment, and repository maintenance. Review