arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2404.04458cs.CV
收藏

JRDB-Social:一个用于理解社会群体内人类互动的上下文与动态的多面机器人数据集

JRDB-Social: A Multifaceted Robotic Dataset for Understanding of Context and Dynamics of Human Interactions Within Social Groups

Monash University(莫纳什大学)

Simindokht Jahangard, Zhixi Cai, Shiki Wen, Hamid Rezatofighi

AI总结:

本文提出JRDB-Social数据集,通过个体属性、群体内互动和社会群体上下文三层标注扩展JRDB,并利用多模态大语言模型评估机器人场景下人类社会行为理解能力。

摘要:

理解人类社会行为在计算机视觉和机器人学中至关重要。个体动作等微观层面的观察存在不足,因此需要一种综合方法,同时考虑个体行为、群体内动态和社会群体层面,以实现全面理解。为解决数据集局限性,本文引入了JRDB-Social,它是JRDB的扩展。JRDB-Social旨在填补跨多样化室内外社会情境中人类理解的空白,并提供三个层面的标注:个体属性、群体内互动和社会群体上下文。该数据集旨在增强我们对机器人应用中人类社会动态的把握。利用近期前沿的多模态大语言模型,我们评估了我们的基准,以探索它们解读人类社会行为的能力。

英文摘要:

Understanding human social behaviour is crucial in computer vision and robotics. Micro-level observations like individual actions fall short, necessitating a comprehensive approach that considers individual behaviour, intra-group dynamics, and social group levels for a thorough understanding. To address dataset limitations, this paper introduces JRDB-Social, an extension of JRDB. Designed to fill gaps in human understanding across diverse indoor and outdoor social contexts, JRDB-Social provides annotations at three levels: individual attributes, intra-group interactions, and social group context. This dataset aims to enhance our grasp of human social dynamics for robotic applications. Utilizing the recent cutting-edge multi-modal large language models, we evaluated our benchmark to explore their capacity to decipher social human behaviour.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑