Measuring and Restoring Constraint Influence in Black-Box LLM Dialogues
In multi-turn dialogues, users can easily impose or revoke constraints, but revocation does not reliably take effect: models keep enacting withdrawn requirements, a failure we call behavioral relapse, or revocation inertia. No existing instrument measures this influence per clause, predicts it before delivery, or repairs it under matched budgets.
sysname{} System
sysname{} system addresses this issue through the following three steps:
- Contract Ledger: Each constraint is paired with an executable checker, revocations are recorded as tombstones, and the net constraint state is compiled ahead of time into a single specification.
- Sequential Ablation Probe: Measures per-clause adherence and incremental behavioral effect.
- Repair Ladder: Operates under token- and attempt-matched budgets. On the dataname{} dataset, using NTasks{} HumanEval tasks and NClauses{} verified checkers, relapse at an 8B operating point climbs from ScaleDelayedMTwo{} to ScaleDelayedMEight{} as constraint load grows, while stronger models sit at floor. Under matched checkers, model, and budget, ahead-of-time compilation significantly reduces relapse against a no-ledger verifier-retry baseline. Blogger's Review: The sysname{} system proposed in this paper effectively measures and restores constraint influence in black-box LLM dialogues, which is a significant research achievement. By using a contract ledger, sequential ablation probe, and repair ladder, the sysname{} system can predict and repair behavioral relapse, improving the reliability and security of LLM dialogues.