Safety‑aligned LLMs usually refuse a harmful request when it is stated plainly, yet they may answer the same request when it is embedded in a role‑play or narrative wrapper. We evaluated this vulnerability in English, modern Chinese and Classical Chinese, finding that Qwen3‑1.7B already yields attack success rates of 89.4% in English, 93.0% in modern Chinese and 95.7% in Classical Chinese.
To study the phenomenon systematically we built the GUISE benchmark. It supplies parallel requests across the three languages, pairs harmful with benign queries, holds out wrapper types for evaluation, and adopts a stricter criterion that counts warn‑then‑answer responses as successful attacks.
Representation analysis shows that merely switching language or register moves the harmful‑request embedding only slightly away from the model’s refusal direction, whereas narrative wrappers shift it much farther, making refusal less likely.
We propose AXIS, which merges preference optimisation with a rotation objective that aligns harmful‑request embeddings with the refusal direction, and a commitment objective that trains the model to refuse outright rather than produce a warn‑then‑answer reply. Across Qwen3‑1.7B, Qwen3‑4B and GLM‑4‑9B, AXIS achieves the highest combined safety and usability scores among the compared methods.
Review