arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2112.03549cs.CV
收藏

GaTector:一个用于注视物体预测的统一框架

GaTector: A Unified Framework for Gaze Object Prediction

Xi’an University of Architecture and Technology(西安建筑科技大学)

Binglu Wang, Tao Hu, Baoshan Li, Xiaojuan Chen, Zhijie Zhang

AI总结:

本文提出统一框架GaTector解决注视物体预测问题,通过SGS特征提取器、Defocus层、能量聚合损失及wUoC新指标,在GOO数据集上三任务均取得优越性能。

摘要:

注视物体预测是一项新提出的任务,旨在发现人类正在注视的物体。它具有巨大的应用意义,但仍然缺乏一个统一的解决方案框架。一个直观的解决方案是将物体检测分支整合到现有的注视预测方法中。然而,以前的注视预测方法通常使用两个不同的网络从场景图像和头部图像中提取特征,这将导致沉重的网络架构,并阻碍每个分支的联合优化。在本文中,我们构建了一个名为 GaTector 的新颖框架,以统一的方式解决注视物体预测问题。特别是,首先提出了一种特定-通用-特定(SGS)特征提取器,利用共享主干网络为场景图像和头部图像提取通用特征。为了更好地考虑输入和任务的特异性,SGS 在共享主干网络之前引入了两个输入特定模块,在共享主干网络之后引入了三个任务特定模块。具体而言,设计了一个新颖的 Defocus 层,用于为物体检测任务生成物体特定特征,而不会丢失信息或需要额外的计算。此外,引入了能量聚合损失来引导注视热图集中在被注视的框上。最后,我们提出了一种新颖的 wUoC 指标,该指标可以揭示框之间的差异,即使它们没有共享重叠区域。在 GOO 数据集上的大量实验验证了我们的方法在所有三个轨道(即物体检测、注视估计和注视物体预测)中的优越性。

英文摘要:

Gaze object prediction is a newly proposed task that aims to discover the objects being stared at by humans. It is of great application significance but still lacks a unified solution framework. An intuitive solution is to incorporate an object detection branch into an existing gaze prediction method. However, previous gaze prediction methods usually use two different networks to extract features from scene image and head image, which would lead to heavy network architecture and prevent each branch from joint optimization. In this paper, we build a novel framework named GaTector to tackle the gaze object prediction problem in a unified way. Particularly, a specific-general-specific (SGS) feature extractor is firstly proposed to utilize a shared backbone to extract general features for both scene and head images. To better consider the specificity of inputs and tasks, SGS introduces two input-specific blocks before the shared backbone and three task-specific blocks after the shared backbone. Specifically, a novel Defocus layer is designed to generate object-specific features for the object detection task without losing information or requiring extra computations. Moreover, the energy aggregation loss is introduced to guide the gaze heatmap to concentrate on the stared box. In the end, we propose a novel wUoC metric that can reveal the difference between boxes even when they share no overlapping area. Extensive experiments on the GOO dataset verify the superiority of our method in all three tracks, i.e. object detection, gaze estimation, and gaze object prediction.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑