arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2406.06239cs.CV

I-MPN:用于高效人在环标注移动眼动追踪数据的归纳消息传递网络

I-MPN: Inductive Message Passing Network for Efficient Human-in-the-Loop Annotation of Mobile Eye Tracking Data

German Research Center for Artificial Intelligence (DFKI)(德国人工智能研究中心);University of Science, VNU-HCM(胡志明市国家大学科学大学);Quy Nhon AI Research and Development Center, FPT Software(FPT软件归仁人工智能研发中心);Max Planck Research School for Intelligent Systems (IMPRS-IS)(马克斯·普朗克智能系统研究院)

Hoang H. Le, Duy M. H. Nguyen, Omair Shahzad Bhatti, Laszlo Kopacsi, Thinh P. Ngo, Binh T. Nguyen, Michael Barz, Daniel Sonntag

AI总结:

本文提出融合目标检测器与空间关系感知I-MPN的人在环学习方法,以少量用户反馈标注实现移动眼动追踪数据中的自动目标识别,并在三个视频序列上验证其性能与标注效率。

摘要:

理解人类在动态环境中如何处理视觉信息,对于心理学和设计以用户为中心的交互至关重要。尽管结合第一人称视角视频与注视信号的移动眼动追踪系统能够提供有价值的洞见,但人工分析这些记录非常耗时。在本研究中,我们提出了一种新颖的以人为中心的学习算法,用于移动眼动追踪场景中的自动化目标识别。我们的方法将目标检测器与具备空间关系感知能力的归纳消息传递网络(I-MPN)无缝集成,利用节点轮廓信息并捕捉目标之间的相关性。这些机制使我们能够学习可泛化到新的目标角度视图的嵌入函数,从而在用户穿行环境时,促进动态情境中的快速适应与高效推理。通过在三个不同视频序列上开展的实验,我们这种基于交互的方法相较固定训练/测试算法表现出显著性能提升,即使在仅使用通过用户反馈收集的、数量少得多的标注样本进行训练时也是如此。此外,我们证明了该方法在数据标注流程中具有出色效率,并超越了以往使用完整目标检测器、将检测器与卷积网络结合,或采用交互式视频分割的交互方法。

英文摘要:

Comprehending how humans process visual information in dynamic settings is crucial for psychology and designing user-centered interactions. While mobile eye-tracking systems combining egocentric video and gaze signals can offer valuable insights, manual analysis of these recordings is time-intensive. In this work, we present a novel human-centered learning algorithm designed for automated object recognition within mobile eye-tracking settings. Our approach seamlessly integrates an object detector with a spatial relation-aware inductive message-passing network (I-MPN), harnessing node profile information and capturing object correlations. Such mechanisms enable us to learn embedding functions capable of generalizing to new object angle views, facilitating rapid adaptation and efficient reasoning in dynamic contexts as users navigate their environment. Through experiments conducted on three distinct video sequences, our interactive-based method showcases significant performance improvements over fixed training/testing algorithms, even when trained on considerably smaller annotated samples collected through user feedback. Furthermore, we demonstrate exceptional efficiency in data annotation processes and surpass prior interactive methods that use complete object detectors, combine detectors with convolutional networks, or employ interactive video segmentation.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑