arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2501.13335cs.CV

Deblur-Avatar:从运动模糊单目视频生成可动画化头像

Deblur-Avatar: Animatable Avatars from Motion-Blurred Monocular Videos

School of Artificial Intelligence and Automation, Huazhong University of Science and Technology(华中科技大学人工智能与自动化学院);S-Lab for Advanced Intelligence, Nanyang Technological University(南洋理工大学先进智能实验室);College of Computing and Data Science, Nanyang Technological University(南洋理工大学计算与数据科学学院);SenseTime Research(商汤科技研究院)

Xianrui Luo, Juewen Peng, Zhongang Cai, Lei Yang, Fan Yang, Zhiguo Cao, Guosheng Lin

AI总结:

该研究针对运动模糊单目视频输入,提出将基于人类运动的运动模糊模型集成到3D高斯平铺的框架,通过优化轨迹和高斯分布重建高质量可动画化头像,实验证明其在渲染质量等方面显著优于现有方法。

摘要:

我们提出了一种新框架,用于从运动模糊的单目视频输入中建模高保真、可动画化的3D人类头像。运动模糊在现实世界动态视频捕捉中很普遍,尤其是在3D人类头像建模中。现有方法要么假设图像输入清晰,无法解决运动模糊导致的细节损失,要么主要考虑相机运动模糊,忽略了在可动画化头像中更常见的人类运动模糊。我们的方法将基于人类运动的运动模糊模型集成到3D高斯平铺(3DGS)中。通过在曝光时间内明确建模人类运动轨迹,我们联合优化轨迹和3D高斯分布以重建清晰、高质量的人类头像。我们采用基于姿势的融合机制来区分移动身体区域,有效优化模糊和清晰区域。在合成和真实世界数据集上的大量实验表明,我们的方法在渲染质量和定量指标上显著优于现有方法,能生成清晰的头像重建,并在具有挑战性的运动模糊条件下实现实时渲染。代码和模型可在指定网址获取。

英文摘要:

We introduce a novel framework for modeling high-fidelity, animatable 3D human avatars from motion-blurred monocular video inputs. Motion blur is prevalent in real-world dynamic video capture, especially due to human movements in 3D human avatar modeling. Existing methods either (1) assume sharp image inputs, failing to address the detail loss introduced by motion blur, or (2) mainly consider blur by camera movements, neglecting the human motion blur which is more common in animatable avatars. Our proposed approach integrates a human movement-based motion blur model into 3D Gaussian Splatting (3DGS). By explicitly modeling human motion trajectories during exposure time, we jointly optimize the trajectories and 3D Gaussians to reconstruct sharp, high-quality human avatars. We employ a pose-dependent fusion mechanism to distinguish moving body regions, optimizing both blurred and sharp areas effectively. Extensive experiments on synthetic and real-world datasets demonstrate that our method significantly outperforms existing methods in rendering quality and quantitative metrics, producing sharp avatar reconstructions and enabling real-time rendering under challenging motion blur conditions. Code and models are available at https://github.com/xianrui-luo/deblur_avatar.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑