展示与打磨：人脸视频修复中参考引导的身份保持
Show and Polish: Reference-Guided Identity Preservation in Face Video Restoration
AI总结：
针对严重退化下人脸视频修复难以保持身份特征的问题，提出IP-FVR方法，利用参考图像和解耦交叉注意力提供身份条件，并结合反馈学习与指数混合策略解决片段内和片段间的身份漂移，显著提升了修复质量与身份一致性。
摘要：
人脸视频修复（FVR）旨在从退化的版本中恢复高质量的人脸视频。传统方法在严重退化时难以保持细粒度的、特定身份的特征，通常生成缺乏个体特征的大众脸。为了应对这些挑战，我们引入了IP-FVR，这是一种新颖的方法，利用高质量的人脸参考图像作为视觉提示，在去噪过程中提供身份条件。IP-FVR使用解耦的交叉注意力机制从参考图像中引入语义丰富的身份信息，确保结果细节丰富且身份一致。对于片段内（24帧内）的身份漂移，我们引入了一种身份保持反馈学习方法，将基于余弦相似度的奖励信号与后缀加权的时间聚合相结合。该方法有效减少了帧序列内的漂移。对于片段间的身份漂移，我们开发了一种指数混合策略，通过在去噪过程中迭代混合前序片段的帧来对齐跨片段的身份。该方法确保了不同片段间身份表示的一致性。此外，我们使用多流负向提示来增强修复过程，引导模型关注相关的面部属性，并最大程度减少低质量或错误特征的生成。在合成和真实数据集上的大量实验表明，IP-FVR在质量和身份保持方面均优于现有方法，展示了其在人脸视频修复实际应用中的巨大潜力。
英文摘要：
Face Video Restoration (FVR) aims to recover high-quality face videos from degraded versions. Traditional methods struggle to preserve fine-grained, identity-specific features when degradation is severe, often producing average-looking faces that lack individual characteristics. To address these challenges, we introduce IP-FVR, a novel method that leverages a high-quality reference face image as a visual prompt to provide identity conditioning during the denoising process. IP-FVR incorporates semantically rich identity information from the reference image using decoupled cross-attention mechanisms, ensuring detailed and identity consistent results. For intra-clip identity drift (within 24 frames), we introduce an identity-preserving feedback learning method that combines cosine similarity-based reward signals with suffix-weighted temporal aggregation. This approach effectively minimizes drift within sequences of frames. For inter-clip identity drift, we develop an exponential blending strategy that aligns identities across clips by iteratively blending frames from previous clips during the denoising process. This method ensures consistent identity representation across different clips. Additionally, we enhance the restoration process with a multi-stream negative prompt, guiding the model's attention to relevant facial attributes and minimizing the generation of low-quality or incorrect features. Extensive experiments on both synthetic and real-world datasets demonstrate that IP-FVR outperforms existing methods in both quality and identity preservation, showcasing its substantial potential for practical applications in face video restoration.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
