arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2602.20193cs.CRcs.AI
收藏

当后门超越触发器:在编码器攻击下扩散模型的语义漂移

When Backdoors Go Beyond Triggers: Semantic Drift in Diffusion Models Under Encoder Attacks

Google(谷歌);Electrical and Computer Engineering Department University of Hawai‘i at Mānoa(电气与计算机工程系,夏威夷大学马诺阿分校)

Shenyang Chen, Liuwan Zhu

AI总结:

研究揭示了在编码器攻击下扩散模型的语义漂移问题,提出SEMAX框架用于量化结构退化,强调几何审计的重要性。

摘要:

标准的文本到图像(T2I)模型后门攻击评估主要测量触发器激活和视觉保真度。我们挑战这一范式,证明编码器侧污染会引发持久的、无触发器的语义破坏,从根本上重塑表示流形。我们追溯这种漏洞到一个几何机制:基于雅可比的分析揭示后门作为低秩、目标中心的变形,放大局部敏感性,导致破坏在语义邻域内相干传播。为了严格量化这种结构退化,我们引入SEMAX(语义对齐与漂移),一个诊断框架,测量内部嵌入漂移和下游功能不匹配。我们的发现,经过扩散和对比范式验证,揭示了编码器污染的深层结构风险,并突显了超越简单攻击成功率的几何审计的必要性。

英文摘要:

Standard evaluations of backdoor attacks on text-to-image (T2I) models primarily measure trigger activation and visual fidelity. We challenge this paradigm, demonstrating that encoder-side poisoning induces persistent, trigger-free semantic corruption that fundamentally reshapes the representation manifold. We trace this vulnerability to a geometric mechanism: a Jacobian-based analysis reveals that backdoors act as low-rank, target-centered deformations that amplify local sensitivity, causing distortion to propagate coherently across semantic neighborhoods. To rigorously quantify this structural degradation, we introduce SEMAD (Semantic Alignment and Drift), a diagnostic framework that measures both internal embedding drift and downstream functional misalignment. Our findings, validated across diffusion and contrastive paradigms, expose the deep structural risks of encoder poisoning and highlight the necessity of geometric audits beyond simple attack success rates.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑