Open-source large language models (OSMs) are achieving near state-of-the-art performance, prompting the integration of text watermarking algorithms directly into their weights for tracing generated text. However, OSMs are subject to post-training modifications, particularly model merging, which can lead to the removal of such watermarks.
This work presents a novel approach to designing OSM watermarks that remain durable against model merging. We introduce Merge-Adversarial Training, an adversarial training algorithm that distills text watermarks into model weights while being robust to subsequent model merging.
Our experiments demonstrate that this method consistently outperforms all baselines. For example, it achieves an increase of 51 percentage points in TPR@1%FPR in SLERP scenarios, with an average improvement of 25 percentage points, all while preserving downstream capabilities. Additionally, we evaluate OSM watermarks against realistic merging scenarios, including common use cases like combining expert capabilities and preventing catastrophic forgetting, using three prominent merging algorithms.
More broadly, our findings indicate that adversarial training is a reliable method for enhancing the durability of OSM watermarks against post-training modifications.
Blogger's Review: The introduction of Merge-Adversarial Training offers a promising avenue for enhancing watermarking techniques in open-source LLMs, particularly in the face of challenges posed by model merging. This approach not only improves watermark effectiveness but also provides a layer of security for downstream model applications, making it a significant contribution to the field.