自适应身份锚定：用于视频人脸交换中合成配对监督的闭环关键帧放置
Adaptive Identity Anchoring: Closed-Loop Keyframe Placement for Synthetic Paired Supervision in Video Face Swapping
浏览论文内容
中文总结 AI 辅助
研究视频人脸交换缺乏自然配对监督问题，提出自适应身份锚定（AIA）方法，通过闭环反馈放置锚定并结合现实参考纹理恢复，还给出可证伪实验，有望解决合成身份漂移及过度平滑皮肤问题。
中文摘要 AI 辅助
视频人脸交换没有自然的配对监督，即不存在一个人面部表演另一个人视频的真实镜头。当前最强的方法DreamID-V的SyncID-Pipe通过在真实剪辑的恰好两帧（第一帧和最后一帧）中替换身份并仅从姿势序列生成其余部分来生成配对。由于姿势不携带换入身份的外观证据，在长剪辑、遮挡和极端姿势变化时，合成身份会有很长的无锚定跨度而漂移，且尚无已发表的消融研究检查锚定数量或放置。我们提出自适应身份锚定（AIA）：（i）将合成器推广到任意锚定集，这在基于帧条件是将其令牌钳制为零噪声的扩散强制式变压器中在架构上是自然的；（ii）通过闭环反馈放置锚定，该闭环将每个生成帧与真实参考身份进行评分，并在得分最差的帧插入图像人脸交换锚定，直到配对通过阈值或用尽预算；（iii）将闭环的判定用作自动数据过滤器。另一个问题，过度平滑皮肤的美颜滤镜外观，有相同的根本原因：微观纹理，与身份一样，在管道的任何目标中都没有被定价。因此，我们将AIA与现实参考纹理恢复相结合：从每个真实帧的非面部区域进行匹配的重新纹理化，从真实镜头进行子身份微观纹理的带分割转移，以及由镜头自身频谱裁判的第二个频谱接受通道。我们认为身份锚定密度是一个可控的质量调节器，并指定了可证伪的实验——漂移与差距曲线、在匹配预算下的均匀与自适应放置、在AIA生成的数据上进行学生训练以及带有人类美颜滤镜研究的纹理消融——来验证或反驳该提议。
英文摘要
Video face swapping has no natural paired supervision: no real footage exists of one person's face performing another person's video. The strongest current answer, DreamID-V's SyncID-Pipe, mints pairs by replacing the identity in exactly two frames of a real clip -- the first and the last -- and regenerating the rest from a pose sequence alone. Pose carries no appearance evidence of the swapped-in identity, so over long clips, occlusions, and extreme pose excursions the synthesized identity has a long unanchored span on which to drift; no published ablation examines anchor count or placement. We propose Adaptive Identity Anchoring (AIA): (i) generalize the synthesizer to arbitrary anchor sets, architecturally natural for diffusion-forcing-style transformers where conditioning on a frame is clamping its tokens to zero noise; (ii) place anchors by a closed feedback loop that scores every generated frame against the real reference identity and inserts an image-face-swapped anchor at the worst-scoring frame until the pair passes a threshold or exhausts a budget; (iii) reuse the loop's verdict as an automatic data filter. A second pathology, the beauty-filter look of over-smoothed skin, has the same root cause: micro-texture, like identity, is priced by none of the pipeline's objectives. We therefore pair AIA with Reality-Referenced Texture Restoration: matched re-graining from each real frame's non-face regions, band-split transfer of sub-identity micro-texture from the real footage, and a second, spectral acceptance channel refereed by the footage's own spectrum. Identity-anchor density, we argue, is a controllable quality dial, and we specify falsifiable experiments -- drift-versus-gap curves, uniform-versus-adaptive placement at matched budgets, student training on AIA-minted data, and texture ablations with a human beauty-filter study -- that would validate or refute the proposal.
