When Watermarks Turn Adversarial: Exposing Vulnerabilities in Document Understanding
Qingzhe Han, Fan Dang, Shengqi Chen, Xinyi Sun, Haoyu Xiong, Lingkun Li, Di Zhang
原文摘要
Visible watermarks are common in digital documents and persist during vision-language model (VLM) processing. We show that they can also serve as adversarial surfaces for document understanding. We propose Watermark-based Adversarial Surface Perturbation (WASP), a white-box attack that optimizes pixels inside fixed watermark strokes without requiring downstream questions, answers, or task-specific losses. We further introduce Balanced Aggregation Loss (BAL), which emphasizes visual token groups that remain highly similar to clean representations. Adversarial watermarks degrade performance substantially more than ordinary watermarks. On DocVQA, WASP achieves attack success rates of 96.81% and 71.93% against Qwen2.5-VL and InternVL2.5; BAL improves over a standard token hinge objective by 9.21 and 2.24 percentage points. On 1,651 OmniDocBench pages, WASP increases MinerU2.5's text edit distance by 0.7748 relative to ordinary watermarks. WASP also transfers within model families and across readers sharing a vision-tower architecture. Attack effectiveness depends on watermark geometry, model architecture, and preprocessing; in structured parsing, disrupted layout and routing can propagate errors downstream. These results expose an underexplored security risk in document understanding systems.
cs.CV