arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2407.18245cs.CVcs.LG
收藏

VGGHeads: 基于大规模合成数据集的3D多头部对齐

VGGHeads: 3D Multi Head Alignment with a Large-Scale Synthetic Dataset

University of Oxford(牛津大学);Piñata Farms;Ukrainian Catholic University(乌克兰天主大学)

Orest Kupyn, Eugene Khvedchenia, Christian Rupprecht

AI总结:

提出VGGHeads,一个由扩散模型生成的大规模合成数据集,用于单步同时进行头部检测和3D网格重建,在真实图像上表现优异。

摘要:

人类头部检测、关键点估计和3D头部模型拟合是许多应用中的基本任务。然而,传统的真实世界数据集常常存在偏差、隐私和伦理问题,并且是在实验室环境中记录的,这使得训练出的模型难以泛化。在这里,我们介绍\method——一个使用扩散模型生成的大规模合成数据集,用于人类头部检测和3D网格估计。我们的数据集包含超过100万张高分辨率图像,每张图像都标注了详细的3D头部网格、面部标志和边界框。利用这个数据集,我们引入了一种新的模型架构,能够从单张图像中单步同时进行头部检测和头部网格重建。通过广泛的实验评估,我们证明了在我们的合成数据上训练的模型在真实图像上取得了强劲的性能。此外,我们数据集的多样性使其适用于广泛的任务,提供了人类头部的通用和全面表示。

英文摘要:

Human head detection, keypoint estimation, and 3D head model fitting are essential tasks with many applications. However, traditional real-world datasets often suffer from bias, privacy, and ethical concerns, and they have been recorded in laboratory environments, which makes it difficult for trained models to generalize. Here, we introduce \method -- a large-scale synthetic dataset generated with diffusion models for human head detection and 3D mesh estimation. Our dataset comprises over 1 million high-resolution images, each annotated with detailed 3D head meshes, facial landmarks, and bounding boxes. Using this dataset, we introduce a new model architecture capable of simultaneous head detection and head mesh reconstruction from a single image in a single step. Through extensive experimental evaluations, we demonstrate that models trained on our synthetic data achieve strong performance on real images. Furthermore, the versatility of our dataset makes it applicable across a broad spectrum of tasks, offering a general and comprehensive representation of human heads.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑