TextGaze：利用文本场景线索提示注视目标估计
TextGaze: Prompting Gaze Target Estimation with Textual Scene Cues
浏览论文内容
中文总结 AI 辅助
研究注视目标估计问题，提出TextGaze统一跨模态架构，利用大型视觉语言模型平衡多分支与简化设计范式，通过冻结编码器提取视觉特征、设计融合模块等进行联合预测，在多数据集上评估有竞争力，为传统设计提供简化替代。
中文摘要 AI 辅助
注视目标估计旨在推断人在场景中的注视位置。主流设计逻辑中，多分支方法需额外监督和标注，而简化设计更注重低级视觉显著性而非真实注视意图。前者导致高标注负担并阻碍领域迁移，后者使预测注意力与实际注视目标不一致。为解决此问题，我们提出TextGaze，一种统一的跨模态架构，利用大型视觉语言模型作为可扩展语义指导来平衡两种设计范式。该模型从冻结编码器提取视觉特征，利用大型视觉语言模型获取与注视对齐的文本线索。我们设计了基于Transformer的融合模块并进行分层文本监督以保留任务语义。轻量级解码头可联合预测注视热图和帧内/外状态。我们在四个主流数据集上评估了该方法，结果显示在关键指标上具有竞争力，且无需额外微调即可实现强大的跨数据集泛化。总体而言，我们为传统设计提供了一种简化替代方案，并突出了大型视觉语言模型作为注视估计可访问辅助指导的潜力。
英文摘要
Gaze target estimation aims to infer the position of a person's gaze within a scene. Within mainstream design logic, multi-branch methods require extra supervision and annotations, while streamlined designs prioritize low-level visual saliency over true gaze intent. The former leads to a high annotation burden and hinders domain transfer, whereas the latter causes misalignment between predicted attention and actual gaze targets. To address this issue, we propose TextGaze, a unified cross-modal architecture that leverages a Large Vision-Language Model (LVLM) as scalable semantic guidance to balance the two design paradigms. The model extracts visual features from a frozen encoder and utilizes an LVLM to obtain gaze-aligned textual cues. We design a transformer-based fusion module with hierarchical text supervision to preserve task semantics. Lightweight decoding heads enable the joint prediction of gaze heatmaps and in-/out-of-frame status. We evaluate our method on four mainstream datasets, and the results show competitive performance across key metrics with robust cross-dataset generalisation without extra fine-tuning. Overall, we provide a streamlined alternative to traditional designs and highlight the potential of LVLMs as accessible auxiliary guidance for gaze estimation.
发表机构
- University of Science and Technology of China（中国科学技术大学）
- Hefei University of Technology（合肥工业大学）
- Institute of Artificial Intelligence, Hefei Comprehensive National Science Center（合肥综合性国家科学中心人工智能研究院）
- Anhui University（安徽大学）
- United Arab Emirates University（阿联酋大学）
机构由 AI 辅助整理，请以论文原文为准。
