AniPortrait：音频驱动的写实肖像动画合成
AniPortrait: Audio-Driven Synthesis of Photorealistic Portrait Animation
AI总结：
提出AniPortrait框架，通过两阶段方法将音频转化为2D面部关键点序列，并利用扩散模型与运动模块生成写实且时间一致的肖像动画，在自然度与可控性上表现优越。
摘要：
本研究提出AniPortrait，一个由音频和参考肖像图像驱动生成高质量动画的新框架。方法分两阶段：首先从音频提取3D中间表示，并将其投影为2D面部关键点序列；随后采用鲁棒的扩散模型结合运动模块，将该关键点序列转换为写实且时间一致的肖像动画。实验结果证明AniPortrait在面部自然度、姿态多样性和视觉质量上具有优越性，提供增强的感知体验。此外，该方法在灵活性和可控性上展现出巨大潜力，可有效应用于面部动作编辑或面部重现等领域。代码和模型权重发布于https://github.com/scutzzj/AniPortrait
英文摘要：
In this study, we propose AniPortrait, a novel framework for generating high-quality animation driven by audio and a reference portrait image. Our methodology is divided into two stages. Initially, we extract 3D intermediate representations from audio and project them into a sequence of 2D facial landmarks. Subsequently, we employ a robust diffusion model, coupled with a motion module, to convert the landmark sequence into photorealistic and temporally consistent portrait animation. Experimental results demonstrate the superiority of AniPortrait in terms of facial naturalness, pose diversity, and visual quality, thereby offering an enhanced perceptual experience. Moreover, our methodology exhibits considerable potential in terms of flexibility and controllability, which can be effectively applied in areas such as facial motion editing or face reenactment. We release code and model weights at https://github.com/scutzzj/AniPortrait
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
