In the realm of Large Language Model (LLM)-assisted software security, the boundary between vulnerability analysis terminology and misuse-related terms is often blurred. Existing safety and cybersecurity evaluations frequently compare unrelated model families, conflating safety behavior with architectural, scale, training data, and deployment differences. To address this, we examine safety states: whether refusal behavior remains intact (Aligned) or has been ablated (Abliterated) within same-lineage models. We investigate how this safety state affects defensive utility across software-security workflows.
We compare aligned instruction-tuned models with publicly released refusal-ablated descendants from two model families, Gemma and Qwen. We evaluate Aligned and Abliterated states on vulnerability detection, CWE attribution, vulnerable-line localization, root-cause localization, and executable patch validation. Furthermore, we treat prompt wording as a controlled framing dimension: prompts begin with neutral code-review language, add authorization context, and vary the density of cybersecurity terminology.
In a Gemma-based Java/Vul4J repair-validation study, Abliterated achieved higher early-stage validation rates, with 67.8%, 65.0%, and 32.8% of patches judged usable, successfully applied, and successfully compiled, respectively, compared to 29.9%, 24.9%, and 9.0% for Aligned. In the Qwen pair, Abliterated improved localization performance, increasing line-level F1 from 2.08% to 3.91% and Top-1 accuracy from 4.10% to 6.95%. These findings suggest that evaluations of LLM-based security assistants should jointly measure whether models respond, whether their usable responses are correct, and whether their outputs remain actionable across the engineering workflow.
Blogger's Review: This paper provides an in-depth comparison of same-lineage models and explores how the ablation of refusal behavior enhances vulnerability detection effectiveness. Its findings are significant for both theoretical understanding and practical applications in software security. Future research could further investigate the impact of different model architectures on security performance.