EgoExoLearn：一个用于桥接真实世界中程序性活动的异步自我中心与外部中心视角的数据集
EgoExoLearn: A Dataset for Bridging Asynchronous Ego- and Exo-centric View of Procedural Activities in Real World
AI总结：
本文提出EgoExoLearn数据集，通过模拟人类示范跟随过程，提供120小时的自我中心与示范视频、注视数据及多模态标注，并建立跨视角关联、动作规划和技能评估基准，以促进AI智能体在真实世界中通过观察人类进行学习。
摘要：
能够将他人的活动映射到自己的视角中是人类的一项基本技能，甚至在很小的时候就已具备。为了向理解这种人类能力迈出一步，我们引入了EgoExoLearn，这是一个模拟人类示范跟随过程的大规模数据集，其中个体在示范视频的指导下执行任务并记录自我中心视频。着眼于日常辅助和专业支持中的潜在应用，EgoExoLearn包含跨越120小时的自我中心与示范视频数据，这些数据采集于日常生活场景和专业实验室。伴随视频，我们记录了高质量的注视数据，并提供了详细的多模态标注，为建模从不同视角桥接异步程序性动作的人类能力构建了平台。为此，我们提出了跨视角关联、跨视角动作规划和跨视角参照技能评估等基准，并进行了详细分析。我们期望EgoExoLearn能成为桥接跨视角动作的重要资源，从而为创造能够在真实世界中通过观察人类进行无缝学习的AI智能体铺平道路。代码和数据可在：https://github.com/OpenGVLab/EgoExoLearn 获取。
英文摘要：
Being able to map the activities of others into one's own point of view is one fundamental human skill even from a very early age. Taking a step toward understanding this human ability, we introduce EgoExoLearn, a large-scale dataset that emulates the human demonstration following process, in which individuals record egocentric videos as they execute tasks guided by demonstration videos. Focusing on the potential applications in daily assistance and professional support, EgoExoLearn contains egocentric and demonstration video data spanning 120 hours captured in daily life scenarios and specialized laboratories. Along with the videos we record high-quality gaze data and provide detailed multimodal annotations, formulating a playground for modeling the human ability to bridge asynchronous procedural actions from different viewpoints. To this end, we present benchmarks such as cross-view association, cross-view action planning, and cross-view referenced skill assessment, along with detailed analysis. We expect EgoExoLearn can serve as an important resource for bridging the actions across views, thus paving the way for creating AI agents capable of seamlessly learning by observing humans in the real world. Code and data can be found at: https://github.com/OpenGVLab/EgoExoLearn
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
