arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2406.01079cs.CVcs.AI
收藏

对象感知的以自我为中心的在线动作检测

Object Aware Egocentric Online Action Detection

Yonsei University(延世大学)

Joungbin An, Yunsu Park, Hyolim Kang, Seon Joo Kim

AI总结:

针对在线动作检测方法未充分利用以自我为中心视频独特视角的问题,提出对象感知模块,将特定先验集成到现有OAD框架中,在Epic-Kitchens 100数据集上验证显示能无缝提升性能且开销最小。

摘要:

Ego4D、EPIC-Kitchens和Ego-Exo4D等以自我为中心的视频数据集的进展丰富了第一人称人类交互的研究,这对于增强现实和辅助生活应用至关重要。尽管取得了这些进展,当前在流媒体视频中高效检测动作的在线动作检测(OAD)方法主要为非自我中心视角设计,因此未能利用以自我为中心视频固有的独特视角。为了解决这一差距,我们引入了一个对象感知模块,将以自我为中心的特定先验集成到现有OAD框架中,增强第一人称视频片段的解释能力。利用对象特定细节和时间动态,我们的模块提高了检测动作时的场景理解能力。在Epic-Kitchens 100数据集上进行了广泛验证,我们的工作可以无缝集成到现有模型中,开销最小,并带来一致的性能提升,标志着将动作检测系统适应以自我为中心视频分析的重要一步。

英文摘要:

Advancements in egocentric video datasets like Ego4D, EPIC-Kitchens, and Ego-Exo4D have enriched the study of first-person human interactions, which is crucial for applications in augmented reality and assisted living. Despite these advancements, current Online Action Detection methods, which efficiently detect actions in streaming videos, are predominantly designed for exocentric views and thus fail to capitalize on the unique perspectives inherent to egocentric videos. To address this gap, we introduce an Object-Aware Module that integrates egocentric-specific priors into existing OAD frameworks, enhancing first-person footage interpretation. Utilizing object-specific details and temporal dynamics, our module improves scene understanding in detecting actions. Validated extensively on the Epic-Kitchens 100 dataset, our work can be seamlessly integrated into existing models with minimal overhead and bring consistent performance enhancements, marking an important step forward in adapting action detection systems to egocentric video analysis.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑