Large Vision-Language Model (LVLM) inference in cloud-edge environments allows efficient deployment by distributing computation between edge devices and cloud servers. However, this process exposes a new attack surface due to the transmission of intermediate vision tokens from the edge to the cloud. This paper investigates vision token manipulation attacks (VTM-Attack) under a black-box man-in-the-middle scenario, where an adversary intercepts and manipulates a subset of transmitted vision tokens under budget constraints.
We propose four naive attack strategies and an optimization-based token selection method. Experiments across six state-of-the-art LVLMs (ranging from 3B to 72B parameters) and four benchmarks reveal that manipulating just 10% of vision tokens can reduce accuracy by up to 88.31%. These findings expose a critical vulnerability in cloud-edge LVLM inference.
Blogger's Review: This paper highlights serious security vulnerabilities in cloud-edge inference, particularly during the transmission of vision tokens. Attackers can significantly degrade model performance through relatively simple manipulation techniques, providing crucial insights for future security mechanisms.