arXiv 2601.072912026-01-12cs.CVcs.AI

基于前缀微调的视觉语义自适应水印

A Visual Semantic Adaptive Watermark grounded by Prefix-Tuning for Large Vision-Language Model

The Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州））；The Hong Kong University of Science and Technology（香港科技大学）；Zhejiang University（浙江大学）

Qi Zheng, Shuliang Liu, Yu Huang, Sihang Jia, Jungang Li, Lyuhao Chen, Junhao Chen, Hanqian Li, Aiwei Liu, Yibo Yan, Xuming Hu

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

本文提出VISA-Mark，一种基于前缀微调的视觉语义自适应水印方法，通过动态视觉证据权重提升视觉与语义保真度，实现高效且可靠的多模态水印技术。

摘要：

水印技术已成为大型视觉-语言模型（LVLMs）中内容追溯和知识产权保护的关键解决方案。然而，视觉无关的水印会引入视觉无关的标记并破坏视觉基础，因为它们强制执行无差别伪随机偏置；而一些语义感知的方法由于拒绝采样导致推理延迟过高。在本文中，我们提出了VIsual Semantic Adaptive Watermark（VISA-Mark），一种新的框架，该框架在严格保持视觉保真度的同时嵌入可检测信号。我们的方法采用了一个轻量级、高效训练的前缀调谐器来提取动态视觉证据权重，这些权重根据视觉输入对候选标记的证据支持进行量化。这些权重指导一个自适应词汇分区和logits扰动机制，将水印强度专门集中在视觉支持的标记上。通过主动将水印与视觉证据对齐，VISA-Mark有效保持了视觉保真度。实验结果证实，VISA-Mark在视觉一致性（Chair-I）方面比传统方法提高了7.8%，并在语义保真度方面表现更优。该框架在保持高检测准确性（96.88% AUC）和鲁棒攻击抵御能力（99.3%）的同时，不牺牲推理效率，有效建立了可靠性保护多模态水印的新标准。

英文摘要：

Watermarking has emerged as a pivotal solution for content traceability and intellectual property protection in Large Vision-Language Models (LVLMs). However, vision-agnostic watermarks introduce visually irrelevant tokens and disrupt visual grounding by enforcing indiscriminate pseudo-random biases, while some semantic-aware methods incur prohibitive inference latency due to rejection sampling. In this paper, we propose the VIsual Semantic Adaptive Watermark (VISA-Mark), a novel framework that embeds detectable signals while strictly preserving visual fidelity. Our approach employs a lightweight, efficiently trained prefix-tuner to extract dynamic Visual-Evidence Weights, which quantify the evidentiary support for candidate tokens based on the visual input. These weights guide an adaptive vocabulary partitioning and logits perturbation mechanism, concentrating watermark strength specifically on visually-supported tokens. By actively aligning the watermark with visual evidence, VISA-Mark effectively maintains visual fidelity. Empirical results confirm that VISA-Mark outperforms conventional methods with a 7.8% improvement in visual consistency (Chair-I) and superior semantic fidelity. The framework maintains highly competitive detection accuracy (96.88% AUC) and robust attack resilience (99.3%) without sacrificing inference efficiency, effectively establishing a new standard for reliability-preserving multimodal watermarking.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。