Safety alignment is undoubtedly a crucial step in the training of LLMs. However, it conceptually fails to distinguish between various domains and the level of potential harm of a query, leading to significant complications in fields like cybersecurity, where models should not be constrained by their safety mechanisms to achieve legitimate, authorized operations.
In this work, we share our findings from a large-scale ablation experiment on 24 open-source LLMs, demonstrating that domain-specific ablation is achievable using standard methodologies, exemplified by the 1T-parameter Kimi K2. Building on recent work that shows refusals in LLMs occupy a multi-dimensional subspace, we find that this refusal is also widely distributed across layers, particularly in trillion-parameter MoE architectures, aiming to capture the part representing harmful concepts in the cybersecurity domain exclusively.
We also investigate the correlation between model features and the effects of domain-specific ablation, identifying that the type of safety training and architecture are the most reliable predictors. Finally, we classify models into 3 tiers of ablation susceptibility and propose a set of conjectures explaining why a particular effect from this intervention might be observed in a given model.
Blogger's Review: This paper delves into the limitations of safety alignment in cybersecurity, emphasizing the importance of domain-specific ablation. Through the analysis of multiple models, the study reveals the significant impact of safety training and architecture on model performance, prompting a reevaluation of how to enhance model utility while maintaining safety.