ZebraPose: 仅使用合成数据进行斑马检测与姿态估计
ZebraPose: Zebra Detection and Pose Estimation using only Synthetic Data
AI总结：
ZebraPose通过合成数据集实现斑马检测与姿态估计，解决真实数据获取困难和非分布场景下的模型泛化问题。
摘要：
收集和标记大规模真实世界野生动物数据集是不切实际的，成本高，易出错且劳动强度大。对于动物监测任务，作为检测、跟踪和姿态估计，非分布观点（例如空中视角）也是通常需要的，但很少在公开数据集中找到。为了解决这个问题，现有方法使用简单的合成技术生成数据，然后需要策略来弥合合成到现实的差距。因此，真实图像、风格约束、复杂的动物模型或预训练网络往往被利用。相比之下，我们使用一个3D逼真模拟器生成了一个完全合成的数据集，并证明它可以消除检测和估计野生斑马2D姿态的需求。此外，现有使用合成数据的顶部-down 2D姿态估计方法假设检测模型可靠。然而，这些方法在非分布场景中（例如包含野生动物或空中影像）往往失效。我们的方法通过使用同一合成数据集训练两种任务来克服这一问题。通过广泛的基准测试，我们证明了仅从我们的合成数据集上从头开始训练的模型在真实图像上泛化得很好。我们使用多种真实世界和合成数据集、预训练和随机初始化的主干网络以及不同的图像分辨率进行这些工作。代码、结果、模型和数据可在https://zebrapose.is.tue.mpg.de/上找到。
英文摘要：
Collecting and labeling large real-world wild animal datasets is impractical, costly, error-prone, and labor-intensive. For animal monitoring tasks, as detection, tracking, and pose estimation, out-of-distribution viewpoints (e.g. aerial) are also typically needed but rarely found in publicly available datasets. To solve this, existing approaches synthesize data with simplistic techniques that then necessitate strategies to bridge the synthetic-to-real gap. Therefore, real images, style constraints, complex animal models, or pre-trained networks are often leveraged. In contrast, we generate a fully synthetic dataset using a 3D photorealistic simulator and demonstrate that it can eliminate such needs for detecting and estimating 2D poses of wild zebras. Moreover, existing top-down 2D pose estimation approaches using synthetic data assume reliable detection models. However, these often fail in out-of-distribution scenarios, e.g. those that include wildlife or aerial imagery. Our method overcomes this by enabling the training of both tasks using the same synthetic dataset. Through extensive benchmarks, we show that models trained from scratch exclusively on our synthetic data generalize well to real images. We perform these using multiple real-world and synthetic datasets, pre-trained and randomly initialized backbones, and different image resolutions. Code, results, models, and data can be found athttps://zebrapose.is.tue.mpg.de/.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
