沉默是金：利用对抗样本在基于LDM的说话人头生成中消除音频控制
Silence is Golden: Leveraging Adversarial Examples to Nullify Audio Control in LDM-based Talking-Head Generation
AI总结：
针对基于LDM的说话人头生成中参考人像易被音频操控且扰动易被净化的问题，本文提出两阶段方法Silencer，通过置零损失消除音频控制并结合抗净化损失生成鲁棒扰动，实现前瞻性人像隐私保护。
摘要：
基于潜在扩散模型（Latent Diffusion Models, LDM）的说话人头动画技术不断进步，使得生成高度逼真且同步的视频成为可能。这些伪造视频与真实视频难以区分，增加了被滥用于诈骗、政治操纵和虚假信息传播的风险。因此，应对这些伦理问题已成为AI安全领域亟待解决的课题。近期的前瞻性防御研究侧重于通过向人像添加扰动来对抗基于LDM的模型。然而，这些方法在保护参考人像免受高级图像到视频动画攻击方面效果不佳。其局限性体现在两个方面：1）它们无法阻止图像被音频信号操控；2）基于扩散的净化技术可以有效消除保护性扰动。为应对这些挑战，我们提出了Silencer，一种旨在前瞻性保护人像隐私的两阶段方法。首先，提出了一种置零损失（nullifying loss），用于在说话人头生成中忽略音频控制。其次，我们在LDM中应用抗净化损失来优化反演潜在特征，以生成鲁棒的扰动。大量实验证明了Silencer在前瞻性保护人像隐私方面的有效性。我们希望这项工作能够提高AI安全社区对与说话人头生成技术相关的关键伦理问题的关注。代码：https://github.com/yuangan/Silencer。
英文摘要：
Advances in talking-head animation based on Latent Diffusion Models (LDM) enable the creation of highly realistic, synchronized videos. These fabricated videos are indistinguishable from real ones, increasing the risk of potential misuse for scams, political manipulation, and misinformation. Hence, addressing these ethical concerns has become a pressing issue in AI security. Recent proactive defense studies focused on countering LDM-based models by adding perturbations to portraits. However, these methods are ineffective at protecting reference portraits from advanced image-to-video animation. The limitations are twofold: 1) they fail to prevent images from being manipulated by audio signals, and 2) diffusion-based purification techniques can effectively eliminate protective perturbations. To address these challenges, we propose Silencer, a two-stage method designed to proactively protect the privacy of portraits. First, a nullifying loss is proposed to ignore audio control in talking-head generation. Second, we apply anti-purification loss in LDM to optimize the inverted latent feature to generate robust perturbations. Extensive experiments demonstrate the effectiveness of Silencer in proactively protecting portrait privacy. We hope this work will raise awareness among the AI security community regarding critical ethical issues related to talking-head generation techniques. Code: https://github.com/yuangan/Silencer.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
