Resource‑exhaustion attacks against vision‑language models (VLMs) have traditionally assumed a unimodal threat model, treating the image as the sole optimization surface while keeping the user‑visible text prompt fixed. Even recent loop‑centric variants remain confined to this single‑channel view, ignoring the possibility of jointly optimizing across modalities. This paper introduces Joint Pixel-Prompt Optimization (JPPO), the first framework that elevates the visible prompt to a first‑class adversarial variable alongside image perturbations. Under a restricted joint‑input threat model, JPPO performs coupled, stage‑wise optimization over both pixel and prompt spaces, yielding a synergistic cost‑amplification effect that is mechanistically distinct from loop‑dependent failures and shows negligible loop incidence in experiments. We evaluate five open‑source VLM families on MS COCO and ImageNet with an $8/255$ $\ell_{\infty}$ budget. JPPO achieves over $4.6\times$ latency and $5.3\times$ energy amplification on Qwen2.5‑VL‑7B, and $36.6\times$ latency with $32.7\times$ energy amplification on BLIP‑2, surpassing all baselines while requiring far fewer optimization iterations. Ablation studies confirm that the amplification stems from multimodal coordination rather than prompt length or isolated modalities. These findings expose structural blind spots in current VLM defenses and motivate cost‑aware robustness evaluation as a primary security requirement for multimodal deployments.
Review