复制作为防御:通过诱导模型复制来保护图像免受编辑
Copying As Defense: Protecting Images from Editing By Eliciting Model Copying
AI总结:
该研究针对DiT图像编辑器的有害编辑问题,提出CaD防御方法,通过诱导模型复制参考图像而非忽略它,大幅降低编辑成功率,且优化速度更快。
摘要:
基于扩散Transformer(DiT)构建的图像编辑器,如Flux Kontext,可将单张照片转换为令人信服的 manipulated 图像,包括有害及未经同意的编辑。主动防御旨在在图像共享前添加不可感知的扰动以阻止此类编辑。现有针对基于DiT的上下文编辑器的防御方法,会使输出对受保护参考图像的关注度降低,但这会让模型可通过文本及自身生成的标记遵循编辑指令,常生成新的、弱接地的图像,而非阻止编辑。我们发现相反策略更有效:不引导模型忽略参考图像,而是鼓励其复制参考图像。通过受控注意力干预,我们证明仅在少数早期块中将目标标记注意力集中于参考图像,足以使编辑行为崩溃,导致模型复制输入(通常为有损副本),而非执行请求的转换。基于此见解,我们提出复制作为防御(Copying-as-Defense, CaD),其最大化早期Transformer块中每个标记的目标到参考的注意力质量。由于注意力行已归一化,增加参考注意力会抑制分配给文本及生成画布的注意力。将目标限制在早期块还可让我们截断反向传播路径,保持优化成本低廉。在各编辑器及基准测试中,CaD将平均编辑成功率从未保护图像的81.4%降至24.0%,相对降低70.6%;相较于最强基线DeContext(43.0%),相对降低幅度进一步达44.3%。此外,CaD每优化步骤的速度约为可比方法的1.8-2.0倍。
英文摘要:
Image editors built on diffusion transformers (DiTs), such as Flux Kontext, make it possible to turn a single photograph into a convincing manipulated image, including harmful and non-consensual edits. Proactive defenses aim to prevent such manipulations by adding an imperceptible perturbation before an image is shared. Existing defenses for DiT-based in-context editors make the output attend less to the protected reference image. But this leaves the model free to follow the edit instruction using the text and its own generated tokens, often producing a new, weakly grounded image rather than preventing the edit. We find that the opposite strategy is more effective: instead of leading the model to ignore the reference, we encourage it to copy it. Through controlled attention interventions, we show that concentrating target-token attention on the reference image in only a few early blocks is sufficient to collapse the editing behavior, causing the model to reproduce its input, often as a lossy copy, rather than execute the requested transformation. Based on this insight, we introduce Copying-as-Defense (CaD), which maximizes per-token target-to-reference attention mass in early transformer blocks. Since attention rows are normalized, increasing reference attention suppresses the attention allocated to the text and generated canvas. Restricting the objective to early blocks also lets us truncate the backpropagation path, keeping optimization cheap. Across editors and benchmarks, CaD reduces average edit success from 81.4% on unprotected images to 24.0%, a 70.6% relative reduction, and by a further 44.3% relative to DeContext, the strongest baseline (43.0%). CaD is also roughly 1.8-2.0x faster per optimization step than comparable methods.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。