从目光到意义：一种无需训练的统一视觉定位与解释AI智能体
From Gaze to Meaning: A Training-Free AI Agent for Unified Grounding and Explanation
浏览论文内容
中文总结 AI 辅助
本文提出一种无需训练的Gaze Target Agent，利用预训练视觉-语言模型和视觉引导提示，实现目光目标预测、注意力定位与物体识别，在GazeFollow和GazeHOI基准上达到最先进性能。
中文摘要 AI 辅助
理解人类注意力是场景解释的基础，然而现有方法通常依赖缺乏可解释性的重度训练模型。先前的方法难以在没有大量监督的情况下联合推理目光目标、被关注物体和视觉定位。据我们所知，这项工作首次引入了无需训练的Gaze Target Agent (GTA)，用于跨任务的目光引导推理，包括目光目标预测、注意力定位和物体识别。这是通过利用预训练的视觉-语言模型，为其增强视觉引导提示，并采用基于记忆的检索策略处理高不确定性样本，从而在无需额外训练的情况下提升性能。我们使用定量指标和定性结果评估了我们的方法。定量上，我们的方法在GazeFollow和GazeHOI基准上达到了最先进的性能。定性上，我们的智能体提供详细的语义预测，即使地面真值标签错误也能预测正确目标，并且在没有词汇约束的情况下保持灵活性。
英文摘要
Understanding human attention is fundamental for scene interpretation, yet existing approaches often rely on heavily trained models that lack interpretability. Prior methods struggle to jointly reason about gaze targets, attended objects, and visual grounding without extensive supervision. To the best of our knowledge, this work introduces the first training-free Gaze Target Agent (GTA) for gaze-guided reasoning across tasks such as gaze target prediction, attention localization, and object identification. This is achieved by leveraging pretrained vision-language models, augmenting them with visually guided prompts, and employing a memory-based retrieval strategy for high-uncertainty samples to improve performance without additional training. We evaluate our approach using both quantitative metrics and qualitative results. Quantitatively, our method achieves state of the art performance on the GazeFollow and GazeHOI benchmarks. Qualitatively, our agent provides detailed semantic predictions, predicts the correct targets even when ground truth labels are wrong, and remains flexible without vocabulary constraints.
发表机构
- Centre for Vision, Speech and Signal Processing (CVSSP), University of Surrey（萨里大学视觉、语音与信号处理中心）
- Surrey Institute for People-Centred AI, University of Surrey（萨里大学以人为本人工智能研究所）
机构由 AI 辅助整理，请以论文原文为准。
