In‑context learning (ICL) enables large language models to leverage demonstrations without updating their parameters. For interactive agents, demonstrations are trajectories containing reasoning steps, actions, and observations that illustrate how to solve a task. Feeding full trajectories into the prompt consumes a lot of context space and hampers overall performance, while natural‑language summaries force the agent to translate vague guidance into concrete action choices. We therefore investigate whether expert action choices can directly supervise a reusable scoring rule.
We introduce CB‑Agent, extending concept‑based ICL to sequential action selection. Expert trajectories are parsed to extract environment‑action pairs; a concept vector is then fitted by maximum‑likelihood estimation to model the conditional distribution over candidate actions given the current context. This distribution combines the learned concept with frozen language‑model features and prior information.
During ReAct‑style interaction, the agent scores environment actions using this distribution without inserting the full demonstrations into the scoring context. Experiments on TextWorld, ALFWorld, and ScienceWorld show that CB‑Agent achieves the highest cross‑benchmark average across all three model sizes and ranks first on TextWorld and ScienceWorld at each size. ALFWorld results demonstrate that the learned concept contributes beyond prior‑only scoring, and further gains are observed when CB‑Agent is combined with Reflexion. The code is released at https://anonymous.4open.science/r/CB-Agent-DFF9/.
Review: CB‑Agent’s explicit learning of action concepts enables efficient reuse of demonstrations, markedly improving decision‑making of large models in interactive environments and offering a solid foundation for future concept‑driven ICL research.