无跳跃连接的强化解耦人脸替换
Reinforced Disentanglement for Face Swapping without Skip Connection
AI总结：
针对人脸替换中身份泄露和非身份属性保留不足的问题，提出无跳跃连接框架WSC-swap，采用双目标编码器分离属性，结合GAN身份移除损失与3DMM非身份保留损失强化解耦，在多项指标上显著超越现有方法。
摘要：
SOTA人脸替换模型仍存在目标身份（即形状）泄露或目标非身份属性（即背景、头发）在最终结果中未能完全保留的问题。我们证明，这种解耦不充分是由先前模型中普遍采用的两种有缺陷的设计造成的：（1）仅依赖一个压缩编码器同时表示语义层面的非身份面部属性（即姿态）和像素层面的非面部区域细节，这同时满足是矛盾的；（2）高度依赖编码器与最终生成器之间的长跳跃连接，将一定量的目标人脸身份泄露到结果中。为修复这些问题，我们引入了一种名为WSC-swap的新人脸替换框架，该框架去除了跳跃连接，并使用两个目标编码器分别捕获像素层面的非面部区域属性和面部区域的语义非身份属性。为进一步强化目标编码器的解耦学习，我们通过对抗训练（即GAN）采用身份移除损失，并通过先验3DMM模型如[11]采用非身份保留损失。在FaceForensics++和CelebA-HQ上的大量实验表明，我们的结果在一系列丰富的指标上显著优于先前的工作，包括一个此前被完全忽视的用于测量身份一致性的新指标。
英文摘要：
The SOTA face swap models still suffer the problem of either target identity (i.e., shape) being leaked or the target non-identity attributes (i.e., background, hair) failing to be fully preserved in the final results. We show that this insufficient disentanglement is caused by two flawed designs that were commonly adopted in prior models: (1) counting on only one compressed encoder to represent both the semantic-level non-identity facial attributes(i.e., pose) and the pixel-level non-facial region details, which is contradictory to satisfy at the same time; (2) highly relying on long skip-connections between the encoder and the final generator, leaking a certain amount of target face identity into the result. To fix them, we introduce a new face swap framework called 'WSC-swap' that gets rid of skip connections and uses two target encoders to respectively capture the pixel-level non-facial region attributes and the semantic non-identity attributes in the face region. To further reinforce the disentanglement learning for the target encoder, we employ both identity removal loss via adversarial training (i.e., GAN) and the non-identity preservation loss via prior 3DMM models like [11]. Extensive experiments on both FaceForensics++ and CelebA-HQ show that our results significantly outperform previous works on a rich set of metrics, including one novel metric for measuring identity consistency that was completely neglected before.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
