Model merging offers an efficient way to build multi‑task large language models (LLMs) without full retraining, yet parameter interference remains a critical obstacle. Existing methods mainly preserve each expert’s capabilities and alleviate interference, but they rarely exploit the degraded behaviors caused by naive merging (e.g., task arithmetic). CoMerge reformulates merging as a preference‑optimization problem and introduces a self‑supervised, conflict‑driven strategy: defects of naive merging are treated as hard negative samples to create annotation‑free preference pairs. Preference optimization then refines lightweight, tensor‑wise merging coefficients, allowing the model to resolve parameter‑space conflicts while retaining task‑specific abilities. Extensive experiments show an average normalized score of 0.9968 on MergeBench, surpassing all data‑free and data‑driven baselines. On Llama‑3.1‑8B‑Instruct, CoMerge yields notable gains on conflict‑sensitive tasks such as instruction following and safety, and remains highly competitive with full‑parameter fine‑tuning despite optimizing only 1,445 scalar coefficients.
Review