HUI360：用于人机交互预测的360°第一视角数据集及基准模型
HUI360: A 360° Egocentric Dataset and Baselines for Human-Robot Interaction Anticipation
浏览论文内容
中文总结 AI 辅助
该研究推出最大野外人机交互预测数据集HUI360及基准模型，含100万条标注，还发布SSUP-HRI的600万条标注，提供自动标注流水线，实现跨数据集评估，推动人机交互预测研究。
中文摘要 AI 辅助
随着机器人在人类密集环境中日益普及，预测人类意图对于实现主动且符合社会规范的行为至关重要。因此，人机交互的自动预测正成为具身智能体面临的关键感知挑战。为此，我们推出HUI360——目前最大的野外人机交互预测数据集及其基准模型集。该数据集由移动机器人在野外环境中采集，历时3个月、覆盖多种场景，捕捉了路人与用户自然、自发的行为，包含多样化的个体，这种多样性可用于评估和提升交互预测模型的泛化能力。我们设计了一套流水线并共享代码，可对任意360°等矩形视频进行自动交互标注，同时提供手动修正接口。通过该流水线，我们发布了HUI360开放数据集的100万条预处理标注，包括通过最先进计算机视觉方法获得的详细2D姿态、面部关键点和分割掩码，并经人工整理以确保高质量的跟踪与交互标注。此外，我们发布了机器人第一视角采集的原始全景360°图像（仅用于研究目的，符合GDPR要求）。最后，我们建立了交互预测的基准模型，包括该任务的首次跨数据集评估：为此，我们还发布了另一现有野外户外移动机器人数据集（SSUP-HRI）的600万条标注。数据集和代码可在此URL获取。
英文摘要
As robots increasingly operate in human-populated environments, anticipating human intentions is essential for enabling proactive and socially aware behavior. Automatic anticipation of human-robot interactions is thus emerging as a crucial perception challenge for embodied agents. To this end, we introduce HUI360, the largest dataset for human-robot interaction anticipation in the wild and its set of baselines. The dataset was collected from a mobile robot, in the wild, over multiple days within a 3-month period, and in several environments, capturing natural, spontaneous behaviors from both passersby and users, and encompassing a diverse range of individuals. This variety enables evaluating and improving the generalization capabilities of interaction anticipation models. We designed a pipeline and share code for automatic interaction annotation in arbitrary 360-degree equirectangular videos, along with interfaces for manual refinement. Using this pipeline, we release the HUI360 open set of 1M pre-processed annotations, including detailed 2D poses, facial keypoints, and segmentation masks, obtained using state-of-the-art computer vision methods and manually curated to ensure high-quality tracking and interaction annotation. Additionally, we release the raw panoptic 360-degree images captured from the robot's egocentric viewpoint (on demand, for research purpose only in compliance with GDPR). Finally, we establish benchmark baselines for interaction anticipation, including the first cross-dataset evaluations for this task: to this end, we also release 6M annotations for another existing in-the-wild outdoor dataset collected from a mobile robot (SSUP-HRI). Dataset and code can be found at https://hucebot.github.io/hui360.
发表机构
- Inria（法国国家信息与自动化研究所）
- CNRS（法国国家科学研究中心）
- Loria（洛林计算机科学与应用实验室）
- HUCEBOT
- Université Paris-Saclay（巴黎-萨克雷大学）
- CEA（法国原子能和替代能源委员会）
- List
机构由 AI 辅助整理，请以论文原文为准。
