Safety‑aligned large language models are expected to refuse harmful requests while staying useful for benign ones. Most evaluations, however, score only the final response and thus conflate intent understanding with response‑level safety. To address this, we introduce RiskCueBench, a recoverability‑controlled benchmark that first checks whether a model can recover a high‑level intent and then evaluates its safety on the same prompt. RiskCueBench comprises 160 latent intents and 640 abstract non‑operational surface prompts across eight risk categories, covering 25 completed model groups and 32,000 successful outputs with rubric‑adjudicated labels. When conditioning on pairs where both surface forms are recoverable, increased contextual specificity leads to less safe behavior: paired bootstrap estimates show a 0.175 drop in refusal rate and a 0.440 decline in global safety score. Mixed‑effects modeling corroborates a 0.107 rise in unsafe cooperation. These findings indicate that safety behavior can be miscalibrated to surface form even after intent recovery, motivating evaluations that report response behavior conditional on semantic recoverability.
Review