AvatarMakeup：面向3D可动画头部虚拟形象的逼真妆容迁移
AvatarMakeup: Realistic Makeup Transfer for 3D Animatable Head Avatars
AI总结：
针对3D虚拟形象妆容迁移中表情一致性、身份保持和细节控制不足的问题，提出AvatarMakeup方法，利用预训练扩散模型和由粗到细策略，结合Coherent Duplication与Refinement Module，实现高质量且动画一致的妆容迁移。
摘要：
与现实生活中的面部美化类似，3D虚拟形象需要个性化定制以增强视觉吸引力，但这一领域尚未得到充分探索。尽管现有的3D高斯编辑方法可被改造用于面部妆容，但这些方法无法满足实现逼真妆容效果的基本要求：1）在可驱动表情下保持外观一致；2）在整个妆容过程中保持身份不变；3）实现对细节的精确控制。为解决这些问题，我们提出了一种专门的3D妆容方法AvatarMakeup，利用预训练扩散模型从任意个体的单张参考照片中迁移妆容模式。我们采用由粗到细的思路，首先保持外观和身份的一致性，然后细化细节。具体而言，扩散模型被用于生成妆容图像作为监督。由于扩散过程中的不确定性，生成的图像在不同视角和表情下不一致。因此，我们提出了一种Coherent Duplication方法，在确保动态和多视角效果一致性的同时，对目标进行粗略上妆。Coherent Duplication通过重新编码生成妆容图像中的平均面部属性来优化全局UV图。通过查询全局UV图，它可以轻松地从任意视角和表情合成一致的妆容引导，以优化目标虚拟形象。在获得粗略妆容虚拟形象后，我们进一步通过在扩散模型中引入Refinement Module来增强妆容，以实现高质量的妆容效果。实验表明，AvatarMakeup在整个动画过程中实现了最先进的妆容迁移质量和一致性。
英文摘要：
Similar to facial beautification in real life, 3D virtual avatars require personalized customization to enhance their visual appeal, yet this area remains insufficiently explored. Although current 3D Gaussian editing methods can be adapted for facial makeup purposes, these methods fail to meet the fundamental requirements for achieving realistic makeup effects: 1) ensuring a consistent appearance during drivable expressions, 2) preserving the identity throughout the makeup process, and 3) enabling precise control over fine details. To address these, we propose a specialized 3D makeup method named AvatarMakeup, leveraging a pretrained diffusion model to transfer makeup patterns from a single reference photo of any individual. We adopt a coarse-to-fine idea to first maintain the consistent appearance and identity, and then to refine the details. In particular, the diffusion model is employed to generate makeup images as supervision. Due to the uncertainties in diffusion process, the generated images are inconsistent across different viewpoints and expressions. Therefore, we propose a Coherent Duplication method to coarsely apply makeup to the target while ensuring consistency across dynamic and multiview effects. Coherent Duplication optimizes a global UV map by recoding the averaged facial attributes among the generated makeup images. By querying the global UV map, it easily synthesizes coherent makeup guidance from arbitrary views and expressions to optimize the target avatar. Given the coarse makeup avatar, we further enhance the makeup by incorporating a Refinement Module into the diffusion model to achieve high makeup quality. Experiments demonstrate that AvatarMakeup achieves state-of-the-art makeup transfer quality and consistency throughout animation.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
