CoMind：从多视角理解人类协作活动
CoMind: Understanding Collaborative Human Activity from Multiple Minds and Views
AI总结：
研究旨在填补人类协作认知过程研究空白，引入CoMind数据集，整合多模态数据并标注，建立相关基准，有助于开发和评估能建模复杂社会互动及推理人类行为的AI系统。
摘要：
人与人之间的协作是日常生活的基本组成部分，对从家务劳动到专业团队合作等各种目标导向活动的成功至关重要。尽管许多研究集中在建模协调和任务执行上，但支持这种协作的认知过程，特别是心理理论（推断他人心理状态的能力），在自然环境中仍难以研究。为填补这一空白，我们引入了一个新颖的以自我为中心和以他人为中心的视频数据集，捕捉烹饪场景中的真实世界协作。该数据集整合了多视角视频、高质量音频、注视跟踪以及3D场景和物体扫描，并带有对物体的共同关注、社交线索和主体间互动以及主体与物体互动的注释。我们为联合注意力估计、社会条件下的物体互动预测和协作式交接预测建立了基准，促进了对多模态感知、主动协助和协作规划的研究。通过提供时间对齐、注释丰富的多模态数据，CoMind有助于开发和评估能够对复杂社会互动进行建模并在协作环境中推理人类行为的人工智能系统。我们的数据集和基准可在该https网址获取。
英文摘要：
Human-human collaboration is a fundamental aspect of everyday life, essential to success in a wide range of goal-directed activities from household tasks to professional teamwork. While much research has focused on modeling coordination and task execution, the cognitive processes that support such collaboration, particularly Theory of Mind (the ability to infer the mental states of others), remain difficult to study in natural settings. To address this gap, we introduce a novel egocentric and exocentric video dataset capturing real-world collaboration in cooking scenarios. The dataset integrates multi-perspective video, high-quality audio, gaze tracking, and 3D scene and object scans, with annotations for shared attention to objects, social cues and interactions between agents, as well as agent-object interactions. We establish benchmarks for Joint Attention Estimation, Socially Conditioned Object Interaction Anticipation, and Collaborative Handover Prediction, enabling research on multimodal perception, proactive assistance, and collaborative planning. By providing temporally aligned, richly annotated multimodal data, CoMind facilitates the development and evaluation of AI systems capable of modeling complex social interactions and reasoning about human behaviors in collaborative environments. Our dataset and benchmarks are made available at https://comind.ethz.ch/.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
