[CS.AI] CABS+: Efficient and Scalable Model Merging via Conflict-Aware Sparsification and Adaptive Weight Allocation

CABS+ is an efficient and scalable model merging method that aims to address parameter conflicts and knowledge interference issues. Compared to the original CABS method, CABS+ optimizes merging coefficients via Adaptive Weight Allocation (AWA) and an asymmetric fitness function, reducing time complexity and improving overall performance. Additionally, CABS+ proposes the Relative Synergy Score (RSS) to quantify model mergeability and guide model selection.

// Example code
void CABSPlus(Model& model1, Model& model2) {
    // Adaptive Weight Allocation
    AdaptiveWeightAllocation awa(model1, model2);
    // Asymmetric Fitness Function
    AsymmetricFitnessFunction aff(model1, model2);
    // Merge models
    Model mergedModel = MergeModels(model1, model2, awa, aff);
    return mergedModel;
}

Experimental results demonstrate that CABS+ outperforms AdaMerging and WUDIMerging methods in terms of performance, stability, and robustness, while using less GPU memory and merging time. Blogger's Review: CABS+ is a very promising model merging method that effectively addresses parameter conflicts and knowledge interference issues. By using Adaptive Weight Allocation and an asymmetric fitness function, CABS+ not only improves overall performance but also reduces time complexity and GPU memory usage. In the future, CABS+ may become a mainstream method in the model merging field.