BabyView数据集：婴儿和幼儿日常经历的高分辨率第一人称视角视频
The BabyView dataset: High-resolution egocentric videos of infants' and young children's everyday experiences
AI总结：
本文发布868小时、覆盖6个月至3岁儿童家庭场景的高分辨率第一人称视角BabyView数据集，并通过多任务标注与自监督模型评估揭示真实儿童经验数据对类人AI的挑战。
摘要：
人类儿童在样本效率方面远远超过现代机器学习算法，能够以远少于当前模型的数据在关键领域取得高性能。这一“数据差距”既是构建智能人工系统的关键挑战，也是理解人类发展的关键挑战。捕捉儿童经验——即他们的“训练数据”——的第一人称视角视频，是比较人类与模型以及开发算法创新以弥合这一差距的关键要素。然而，目前此类数据集很少，现有数据分辨率低、元数据有限，而且重要的是，仅代表儿童经验的一小部分。在此，我们首次公开发布一个大型发展性第一人称视角视频数据集——BabyView数据集——使用具有大垂直视场的高分辨率相机以及陀螺仪/加速度计数据录制。这个868小时的数据集包含年龄在6个月至3岁之间的儿童在纵向家庭环境中的第一人称视角视频。我们为语音转写、说话人日志和人体姿态估计的评估提供金标准标注，并在这些领域中的每一个领域评估模型。我们训练自监督语言和视觉模型，并评估它们向分布外任务的迁移，包括句法结构学习、物体识别、深度估计和图像分割。尽管每个领域的性能都随数据集规模而提升，但总体性能相对低于模型在精选数据集上训练时的表现，尤其是在视觉领域。我们的数据集是对鲁棒、类人AI系统的一项开放挑战：此类系统如何在与人类相同规模和分布的训练数据上达到人类水平的成功？
英文摘要：
Human children far exceed modern machine learning algorithms in their sample efficiency, achieving high performance in key domains with much less data than current models. This ''data gap'' is a key challenge both for building intelligent artificial systems and for understanding human development. Egocentric video capturing children's experience--their ''training data''--is a key ingredient for comparison of humans and models and for the development of algorithmic innovations to bridge this gap. Yet there are few such datasets available, and extant data are low-resolution, have limited metadata, and importantly, represent only a small set of children's experiences. Here, we provide the first release of a large developmental egocentric video dataset--the BabyView dataset--recorded using a high-resolution camera with a large vertical field-of-view and gyroscope/accelerometer data. This 868 hour dataset includes egocentric videos from children spanning 6 months to 3 years of age in longitudinal, at-home contexts. We provide gold-standard annotations for the evaluation of speech transcription, speaker diarization, and human pose estimation, and evaluate models in each of these domains. We train self-supervised language and vision models and evaluate their transfer to out-of-distribution tasks, including syntactic structure learning, object recognition, depth estimation, and image segmentation. Although performance in each domain scales with dataset size, overall performance is relatively lower than when models are trained on curated datasets, especially in the visual domain. Our dataset stands as an open challenge for robust, human-like AI systems: how can such systems achieve human-levels of success on the same scale and distribution of training data as humans?
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
