一次拍摄，一次对话：从单张图像生成全身说话虚拟形象
One Shot, One Talk: Whole-body Talking Avatar from a Single Image
AI总结：
本文提出从单张图像构建全身说话虚拟形象的流程，利用姿态引导扩散模型生成伪标签，并结合紧耦合3DGS-mesh混合表示与正则化实现逼真、可控的动画效果。
摘要：
构建逼真且可动画化的虚拟形象，目前仍需要数分钟的多视角或单目自旋转视频，而且大多数方法缺乏对手势和表情的精确控制。为突破这一限制，我们致力于解决从单张图像构建全身说话虚拟形象的挑战。我们提出了一种新颖的流程，用以处理两个关键问题：1）复杂的动态建模；2）对新颖手势和表情的泛化。为实现无缝泛化，我们利用最新的姿态引导图像到视频扩散模型生成不完美的视频帧作为伪标签。为克服由不一致且含噪的伪视频带来的动态建模挑战，我们引入了一种紧耦合的3DGS-mesh混合虚拟形象表示，并施加若干关键正则化，以缓解不完美标签造成的不一致。在多样化对象上进行的大量实验表明，我们的方法能够仅凭单张图像创建具有照片级真实感、可精确动画化且富有表现力的全身说话虚拟形象。
英文摘要：
Building realistic and animatable avatars still requires minutes of multi-view or monocular self-rotating videos, and most methods lack precise control over gestures and expressions. To push this boundary, we address the challenge of constructing a whole-body talking avatar from a single image. We propose a novel pipeline that tackles two critical issues: 1) complex dynamic modeling and 2) generalization to novel gestures and expressions. To achieve seamless generalization, we leverage recent pose-guided image-to-video diffusion models to generate imperfect video frames as pseudo-labels. To overcome the dynamic modeling challenge posed by inconsistent and noisy pseudo-videos, we introduce a tightly coupled 3DGS-mesh hybrid avatar representation and apply several key regularizations to mitigate inconsistencies caused by imperfect labels. Extensive experiments on diverse subjects demonstrate that our method enables the creation of a photorealistic, precisely animatable, and expressive whole-body talking avatar from just a single image.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
