通过语言引导的场景上下文感知学习实现鲁棒的自体视觉注意力预测
Robust Egocentric Visual Attention Prediction Through Language-guided Scene Context-aware Learning
AI总结：
本文提出语言引导的场景上下文感知学习框架，通过上下文感知器和双训练目标提升自体视频中注意力预测的鲁棒性与准确性。
摘要：
随着分析自体视频需求的增长，自体视觉注意力预测，即预测摄像机佩戴者会关注的位置，越来越受到关注。然而，由于动态自体场景的固有复杂性和模糊性，这一任务仍然具有挑战性。受证据启发，即场景上下文信息在调节人类注意力中起关键作用，本文提出了一种语言引导的场景上下文感知学习框架，用于实现鲁棒的自体视觉注意力预测。我们首先设计了一个上下文感知器，该感知器基于基于语言的场景描述来总结自体视频，生成上下文感知的视频表示。然后，我们引入了两个训练目标：1）鼓励框架聚焦于目标兴趣点区域；2）抑制来自不相关区域的干扰，这些区域不太可能吸引第一人称注意力。在Ego4D和Aria Everyday Activities (AEA)数据集上的大量实验表明，我们的方法有效，实现了在多样、动态的自体场景中的最佳性能和增强的鲁棒性。
英文摘要：
As the demand for analyzing egocentric videos grows, egocentric visual attention prediction, anticipating where a camera wearer will attend, has garnered increasing attention. However, it remains challenging due to the inherent complexity and ambiguity of dynamic egocentric scenes. Motivated by evidence that scene contextual information plays a crucial role in modulating human attention, in this paper, we present a language-guided scene context-aware learning framework for robust egocentric visual attention prediction. We first design a context perceiver which is guided to summarize the egocentric video based on a language-based scene description, generating context-aware video representations. We then introduce two training objectives that: 1) encourage the framework to focus on the target point-of-interest regions and 2) suppress distractions from irrelevant regions which are less likely to attract first-person attention. Extensive experiments on Ego4D and Aria Everyday Activities (AEA) datasets demonstrate the effectiveness of our approach, achieving state-of-the-art performance and enhanced robustness across diverse, dynamic egocentric scenarios.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
