NeFut Logo NeFut
Admin Login

[CS.AI] SiLR: Structure-Preserving Admission and Process Reward for LLM Tool Agents

Published at: 2026-09-07 22:00 Last updated: 2026-09-08 00:37
#AI #Machine Learning #LLM

For LLM tool agents, the runtime admission gate is usually implemented as a filter. In a ReAct loop a rejected proposal is retried at the same state, making the gate a search operator over the proposal stream; its admission criterion determines which trajectories are reachable. This work studies post‑violation recovery admission, where progress must be allowed while the system remains in violation, and identifies the scalar projection trap: an aggregate‑score gate accepts a locally improving proposal but commits the trajectory to a plateau. SiLR instead shadow‑executes each proposal and admits it under a product order on branch‑level violation state (overloaded‑branch support and per‑branch severity). We prove that no scalar surrogate can soundly represent this order, so the failure is representational rather than a matter of threshold tuning. Experiments on Gym‑ANM scenarios show SiLR recovers 21/21 multi‑action episodes, versus 0/21 for a terminal gate and 9/21 for the best scalar gate, with significance across the full 24‑scenario benchmark. The terminal‑versus‑structured dichotomy holds across three model families and CityLearn. Because admission relies on deterministic simulation, the LLM lies outside the trust boundary: a magnitude‑redistribution attack defeats both scalar and support‑only baselines, and only the full per‑branch predicate contains it. With two constraint families active, every tested scalar projection admits physically unsafe actions; support‑only admits the largest unsafe fraction (63.2% of 42,410), while the product order admits none. In the hardest dual‑family traces, scalar gates recover only through that unsafe class. When reused as a GRPO process reward, SiLR outperforms its count projection in every mined scenario and is the only reward whose ungated policy exceeds the untrained base (0.844 vs 0.778). Scalar projection loses the violation geometry at both design points; only the full product order is structurally sufficient.

Review

Original Source: https://arxiv.org/abs/2609.04629

[h] Back to Home