arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2412.08101cs.CVcs.LG
收藏

生成式动物园(Generative Zoo)

Generative Zoo

Max Planck Institute for Intelligent Systems(马克斯·普朗克智能系统研究所);IMATI-CNR(意大利国家研究委员会IMATI研究所)

Tomasz Niewiadomski, Anastasios Yiannakidis, Hanz Cuevas-Velasquez, Soubhik Sanyal, Michael J. Black, Silvia Zuffi, Peter Kulits

AI总结:

针对动物3D姿态形状估计训练数据难获取的问题,提出用条件图像生成模型构建合成数据集GenZoo,仅用其训练的回归器就在真实基准上达到SOTA。

摘要:

基于模型的图像三维动物姿态与形状估计能够实现动物行为的计算建模。用于该任务的训练模型需要大量带有精确姿态与形状标注的标记图像数据。然而,采集这类数据需要使用多视角或基于标记的动作捕捉系统,这类系统难以应用于野外原位的野生动物,也无法在广泛的动物物种中规模化推广。\n部分研究试图通过人工2D标注对单张真实世界图像进行伪标记,再针对这些标签优化3D参数,以此解决训练数据获取的难题。尽管该方法可以生成与轮廓对齐的样本,但由于单目拟合问题的不适定性,得到的姿态与形状参数通常不符合实际。\n为规避真实世界数据的模糊性,另有研究设计了复杂的合成数据生成流程,利用游戏引擎和艺术家设计的3D资产集合。这类引擎能生成完美的真值标注,但往往缺乏视觉真实感,且需要大量人工工作才能适配新物种或新环境。\n受这些不足的驱动,我们提出了一种合成数据生成的替代方法:使用条件图像生成模型进行渲染。我们提出的流程能够为多种哺乳动物类四足动物采样多样化的姿态与形状,并生成带有对应真值姿态与形状参数的真实感图像。为验证我们方法的可扩展性,我们推出了GenZoo——一个包含100万张不同个体图像的合成数据集。我们在GenZoo上训练了一个3D姿态与形状回归器,尽管仅在合成数据上训练,该模型在真实世界动物姿态与形状估计基准上取得了当前最优性能。数据集可访问:https://genzoo.is.tue.mpg.de

英文摘要:

The model-based estimation of 3D animal pose and shape from images enables computational modeling of animal behavior. Training models for this purpose requires large amounts of labeled image data with precise pose and shape annotations. However, capturing such data requires the use of multi-view or marker-based motion-capture systems, which are impractical to adapt to wild animals in situ and impossible to scale across a comprehensive set of animal species. Some have attempted to address the challenge of procuring training data by pseudo-labeling individual real-world images through manual 2D annotation, followed by 3D-parameter optimization to those labels. While this approach may produce silhouette-aligned samples, the obtained pose and shape parameters are often implausible due to the ill-posed nature of the monocular fitting problem. Sidestepping real-world ambiguity, others have designed complex synthetic-data-generation pipelines leveraging video-game engines and collections of artist-designed 3D assets. Such engines yield perfect ground-truth annotations but are often lacking in visual realism and require considerable manual effort to adapt to new species or environments. Motivated by these shortcomings, we propose an alternative approach to synthetic-data generation: rendering with a conditional image-generation model. We introduce a pipeline that samples a diverse set of poses and shapes for a variety of mammalian quadrupeds and generates realistic images with corresponding ground-truth pose and shape parameters. To demonstrate the scalability of our approach, we introduce GenZoo, a synthetic dataset containing one million images of distinct subjects. We train a 3D pose and shape regressor on GenZoo, which achieves state-of-the-art performance on a real-world animal pose and shape estimation benchmark, despite being trained solely on synthetic data. https://genzoo.is.tue.mpg.de

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑