arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2502.10982cs.CV
收藏

TEASER:用于表情重建的Token增强空间建模

TEASER: Token Enhanced Spatial Modeling for Expressions Reconstruction

International Digital Economy Academy(国际数字经济学院);Peking University, Shenzhen Graduate School(北京大学深圳研究生院)

Yunfei Liu, Lei Zhu, Lijian Lin, Ye Zhu, Ailing Zhang, Yu Li

AI总结:

针对3D人脸重建中细粒度表情捕捉的不足,提出TEASER方法,通过多尺度tokenizer提取面部信息并结合神经渲染器提供几何指导,引入姿态相关关键点损失,显著提升了表情重建质量并达到SOTA。

摘要:

从单张非受限图像中进行3D人脸重建是以人为中心的计算机视觉任务中的一项关键任务。虽然现有方法能恢复准确的人脸形状,但在细粒度表情捕捉方面仍有很大改进空间。当前方法在处理不规则嘴型、夸张表情和不对称面部动作时存在困难。我们提出了TEASER(用于表情重建的Token增强空间建模),以解决这些挑战并提升3D人脸几何性能。TEASER解决了现有方法的两个主要局限:用于自重建的光度损失不足,以及细微表情定位不准确。我们引入多尺度tokenizer来提取面部外观信息。结合神经渲染器,这些token为表情重建提供精确的几何指导。此外,TEASER结合了姿态相关的关键点损失,以进一步提升几何性能。我们的方法不仅显著提升了表情重建质量,还提供了可解释的token,适用于多种下游应用,如照片级真实感人脸视频驱动、表情迁移和身份替换。在多个数据集上的定量和定性实验结果表明,TEASER在精确表情重建方面达到了最先进的性能。

英文摘要:

3D facial reconstruction from a single in-the-wild image is a crucial task in human-centered computer vision tasks. While existing methods can recover accurate facial shapes, there remains significant space for improvement in fine-grained expression capture. Current approaches struggle with irregular mouth shapes, exaggerated expressions, and asymmetrical facial movements. We present TEASER (Token EnhAnced Spatial modeling for Expressions Reconstruction), which addresses these challenges and enhances 3D facial geometry performance. TEASER tackles two main limitations of existing methods: insufficient photometric loss for self-reconstruction and inaccurate localization of subtle expressions. We introduce a multi-scale tokenizer to extract facial appearance information. Combined with a neural renderer, these tokens provide precise geometric guidance for expression reconstruction. Furthermore, TEASER incorporates a pose-dependent landmark loss to further improve geometric performances. Our approach not only significantly enhances expression reconstruction quality but also offers interpretable tokens suitable for various downstream applications, such as photorealistic facial video driving, expression transfer, and identity swapping. Quantitative and qualitative experimental results across multiple datasets demonstrate that TEASER achieves state-of-the-art performance in precise expression reconstruction.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑