This work examines how a fixed‑parameter Transformer can learn algorithms from supplied examples when its actions shape the next evidence in a closed‑loop interaction. We name this setting Agentic In‑Context Learning (Agentic ICL). An interactive learner is formalized by a state, an update rule, and a decision rule, covering active learning, bandits, and online reinforcement learning. To demonstrate expressivity, we construct causal Transformers that approximate these algorithms over a finite horizon and derive an upper bound on the resulting return loss. We then characterize conditions under which optimizing process rewards preserves behavior that maximizes the final outcome. Distillation experiments show that Transformers can learn to execute three representative recursions. Subsequently we train models using only outcome or process rewards, without oracle action or state targets, akin to post‑training reinforcement learning. Different reward signals lead to distinct decision policies. Aligning process supervision with outcome‑optimal action preferences shifts the learned decisions, and these behavioral differences persist in a pretrained language model across textual paraphrases.
Review: The paper offers a coherent theoretical framework and empirical evidence, revealing the potential of Transformers to acquire algorithmic behavior in closed‑loop settings while highlighting the pivotal role of reward design in shaping agent behavior.