Classifying cybersecurity vulnerabilities using the Common Weakness Enumeration (CWE) taxonomy poses challenges due to extreme class imbalance and strong hierarchical dependencies among weakness categories. While oversampling techniques like SMOTE and ADASYN are commonly used to mitigate class imbalance, their effectiveness in hierarchical CWE text classification remains largely unexplored.
This paper proposes a Hierarchy-Aware RoBERTa framework that integrates CWE structural information through learnable parent-class embeddings, preserving taxonomic consistency. Our experiments demonstrate that synthetic interpolation in high-dimensional embedding spaces violates inherent parent-child constraints of the CWE hierarchy, providing only marginal benefits for classical ML models while consistently degrading deep learning architectures.
Evaluated on a CWE Research Concept dataset, the proposed model achieves a weighted F1-score of 0.76 without data augmentation, outperforming all baselines with notable gains on minority classes, including the Class category whose F1-score improved from 0.40 to 0.60 over the BERT baseline. Our results suggest that hierarchy-aware representation learning is a more principled alternative to oversampling for structured vulnerability classification.
Blogger's Review: This paper introduces a novel approach through hierarchy-aware representation learning, offering promising results in addressing class imbalance, especially within deep learning architectures. This research opens avenues for further exploration and application in related fields.