LLM agents solve complex tasks by invoking reusable skills. As the skill library grows, loading all skills incurs heavy token and runtime overhead. Conventional flat semantic retrieval treats each skill as an independent text entry, ignoring dependencies, execution order, state transitions, and data‑flow constraints, which hampers both efficiency and accuracy.
SPEM introduces a lightweight structure‑aware procedural memory framework. It consists of Atomic Skill Memory, which stores individual skill descriptions and implementations, and Skill‑DAG Memory, which records successful compositions of skills as directed acyclic graphs capturing dependency and execution order. When a new task arrives, SPEM first tries to match an existing high‑confidence Skill‑DAG template; if none is suitable, it retrieves atomic skills and passes them to Skill‑GNN for state‑aware and structure‑aware selection.
Skill‑GNN encodes the current environment state, input‑output signatures of candidate skills, and the DAG topology into node features, then uses a graph neural network to predict the most appropriate next skill and its arguments. After a successful execution, the whole trace is written back into Skill‑DAG Memory as a new reusable template, allowing the procedural memory to grow over time.
Evaluations on ALFWorld and SkillsBench demonstrate that SPEM achieves a superior effectiveness‑efficiency trade‑off: 87.1% success on ALFWorld and 38.38% reward on SkillsBench, while reducing token consumption and inference latency compared with strong baselines.
Review