Safety alignment aims to teach large language models to recognize harmful requests and reject risky instructions. Yet aligned models can still fail when malicious intent is hidden within seemingly benign contexts.
Addressing this requires vigilance: the ability to examine a request's underlying intent and assumptions, spotting unusual premises, misleading reasoning, and latent risks before acting.
We introduce cunning questions, which are not necessarily safety‑related but contain deceptive premises, atypical reasoning, or subtle inconsistencies. Training models to navigate these traps is hypothesized to transfer to safety‑critical scenarios.
Experiments across several benchmarks show that adding cunning training improves robustness to out‑of‑distribution jailbreak attacks. The mean attack success rate across nine backbone‑benchmark pairs drops from 17.40% to 15.05%, and subsequent safety fine‑tuning further strengthens performance.
Trace analysis after matched safety fine‑tuning indicates that safety judgments are more likely to intervene before harmful planning begins, effectively cutting off risk early.
A conditional theoretical analysis characterizes when invariances learned from cunning data can transfer to safety‑related inputs, offering a formal justification.
In summary, cunning data can enhance model vigilance and serve as a valuable complement to conventional safety alignment.
Review