Make-Your-Anchor：一个基于扩散的2D数字人生成框架
Make-Your-Anchor: A Diffusion-based 2D Avatar Generation Framework
AI总结：
本文提出Make-Your-Anchor框架，仅需一分钟视频即可生成具有全身动作的主播风格视频，通过结构引导扩散模型和两阶段训练策略实现动作与外观绑定，并采用3D U-Net扩展和批次重叠去噪模块生成任意时长视频。
摘要：
尽管基于说话人头像的数字人生成方案取得了显著进展，但直接生成带有全身动作的主播风格视频仍然具有挑战性。在本研究中，我们提出了Make-Your-Anchor，这是一个新颖的系统，仅需个人的一分钟视频片段进行训练，随后即可自动生成具有精确躯干和手部动作的主播风格视频。具体而言，我们在输入视频上微调一个提出的结构引导扩散模型，将3D网格条件渲染为人类外观。我们对扩散模型采用两阶段训练策略，有效地将动作与特定外观绑定。为了生成任意时长的视频，我们将逐帧扩散模型中的2D U-Net扩展为3D风格，且无需额外的训练成本，并提出一个简单而有效的批次重叠时间去噪模块，以绕过推理期间对视频长度的限制。最后，引入了一个新颖的特定身份人脸增强模块，以提高输出视频中面部区域的视觉质量。对比实验证明了该系统在视觉质量、时间连贯性和身份保留方面的有效性和优越性，优于SOTA扩散/非扩散方法。项目页面：https://github.com/ICTMCG/Make-Your-Anchor。
英文摘要：
Despite the remarkable process of talking-head-based avatar-creating solutions, directly generating anchor-style videos with full-body motions remains challenging. In this study, we propose Make-Your-Anchor, a novel system necessitating only a one-minute video clip of an individual for training, subsequently enabling the automatic generation of anchor-style videos with precise torso and hand movements. Specifically, we finetune a proposed structure-guided diffusion model on input video to render 3D mesh conditions into human appearances. We adopt a two-stage training strategy for the diffusion model, effectively binding movements with specific appearances. To produce arbitrary long temporal video, we extend the 2D U-Net in the frame-wise diffusion model to a 3D style without additional training cost, and a simple yet effective batch-overlapped temporal denoising module is proposed to bypass the constraints on video length during inference. Finally, a novel identity-specific face enhancement module is introduced to improve the visual quality of facial regions in the output videos. Comparative experiments demonstrate the effectiveness and superiority of the system in terms of visual quality, temporal coherence, and identity preservation, outperforming SOTA diffusion/non-diffusion methods. Project page: \url{https://github.com/ICTMCG/Make-Your-Anchor}.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
