Boosting Gaze Object Prediction via Pixel-level Supervision from Vision Foundation Model
Xi’an University of Architecture and Technology（西安建筑科技大学）；School of Automation, Northwestern Polytechnical University（西北工业大学自动化学院）；School of Intelligence Science and Technology, Peking University（北京大学智能科学与技术学院）
Yang Jin, Lei Zhang, Shi Yan, Bin Fan, Binglu Wang
Gaze object prediction (GOP) aims to predict the category and location of the object that a human is looking at. Previous methods utilized box-level supervision to identify the object that a person is looking at, but struggled with semantic ambiguity, ie, a single box may contain several items since objects are close together. The Vision foundation model (VFM) has improved in object segmentation using box prompts, which can reduce confusion by more precisely locating objects, offering advantages for fine-grained prediction of gaze objects. This paper presents a more challenging gaze object segmentation (GOS) task, which involves inferring the pixel-level mask corresponding to the object captured by human gaze behavior. In particular, we propose that the pixel-level supervision provided by VFM can be integrated into gaze object prediction to mitigate semantic ambiguity. This leads to our gaze object detection and segmentation framework that enables accurate pixel-level predictions. Different from previous methods that require additional head input or ignore head features, we propose to automatically obtain head features from scene features to ensure the model's inference efficiency and flexibility in the real world. Moreover, rather than directly fuse features to predict gaze heatmap as in existing methods, which may overlook spatial location and subtle details of the object, we develop a space-to-object gaze regression method to facilitate human-object gaze interaction. Specifically, it first constructs an initial human-object spatial connection, then refines this connection by interacting with semantically clear features in the segmentation branch, ultimately predicting a gaze heatmap for precise localization. Extensive experiments on GOO-Synth and GOO-Real datasets demonstrate the effectiveness of our method.
On Boosting Semantic Street Scene Segmentation with Weak Supervision
Eindhoven University of Technology（埃因霍温理工大学）
训练用于语义分割的卷积网络需要每像素的地面真值标签，这非常耗时，因此获得成本很高。因此，在这项工作中，我们研究和开发了一种分层的深度网络架构和相应的语义分割损失，可以从弱监督（如边界框或图像级别标签）以及强逐像素监督中进行训练。我们证明了层次结构和强（每像素）和弱（边界框）标签的同时训练，即使来自单独的数据集，也会不断提高仅针对每像素训练的性能。此外，我们探索了添加弱图像级标签的更具挑战性的情况。我们从巨大的Open Images数据集中收集街景图像和弱标签来生成OpenScapes数据集，并使用这个新的数据集来提高两个已建立的每像素标记数据集Cityscapes和Vistas的分割性能。我们报告在关键街道场景类上的性能提升高达+13.2%mIoU，在512 x 1024分辨率下，Titan V GPU的Cityscapes推理速度为20 fps。我们的网络和OpenScapes数据集与研究社区共享。
Enjoying Information Dividend: Gaze Track-based Medical Weakly Supervised Segmentation
National Engineering Laboratory for Integrated Aero-Space-Ground-Ocean Big Data Application Technology（集成空天地海大数据应用技术国家工程实验室）；School of Computer Science and Engineering, Northwestern Polytechnical University（计算机科学与工程学院，西北工业大学）；Ningbo Institute of Northwestern Polytechnical University（西北工业大学宁波研究院）
Boosting Box-supervised Instance Segmentation with Pseudo Depth
Zhejiang University of Technology（浙江工业大学）；Zhejiang University（浙江大学）；vivo（维沃（vivo））；Northwestern Polytechnical University（西北工业大学）；Swansea University（斯旺西大学）
SP3D: Boosting Sparsely-Supervised 3D Object Detection via Accurate Cross-Modal Semantic Prompts
Fujian Key Laboratory of Sensing and Computing for Smart Cities, Xiamen University（厦门大学福建省智慧城市感知与计算重点实验室）；Key Laboratory of Multimedia Trusted Perception and Efficient Computing, Ministry of Education of China, Xiamen University（厦门大学多媒体可信感知与高效计算教育部重点实验室）
Gaze-DETR: Top-Down Guidance Through Priority Maps for Infrared Weak-Small UAV Detection with DETR
Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；University of Chinese Academy of Sciences（中国科学院大学）；ShanghaiTech University（上海科技大学）；Beijing Institute of Basic Medical Sciences（北京基础医学研究所）
Personal Fixations-Based Object Segmentation with Object Localization and Boundary Preservation
Shanghai Institute for Advanced Communication and Data Science（上海先进通信与数据科学研究院）；Shanghai University（上海大学）；School of Communication and Information Engineering, Shanghai University（上海大学通信与信息工程学院）；School of Computer Science and Engineering, Nanjing University of Science and Technology（南京理工大学计算机科学与工程学院）；Stony Brook University（石溪大学）
Boosting Low-Data Instance Segmentation by Unsupervised Pre-training with Saliency Prompt
Northwestern Polytechnical University（西北工业大学）；Zhejiang Lab（之江实验室）；Inception Institute of Artificial Intelligence（起源人工智能研究院）；University of Chinese Academy of Sciences（中国科学院大学）；NetEase（网易）；Huazhong University of Science and Technology（华中科技大学）
DualGazeNet: A Biologically Inspired Dual-Gaze Query Network for Salient Object Detection
School of Computation, Information and Technology, Technical University of Munich（计算信息与技术学院，慕尼黑技术大学）；Department of Informatics, Technical University of Munich（信息学院，慕尼黑技术大学）；University Research and Innovation Center, Obuda University（奥布达大学研究与创新中心）；State Key Laboratory for Novel Software Technology, Nanjing University Suzhou Campus（新型软件技术国家重点实验室，南京大学苏州校区）；School of Science and Technology, Nanjing University Suzhou Campus（科学与技术学院，南京大学苏州校区）；Department of Computer Science and Technology, Tsinghua University（计算机科学与技术系，清华大学）
Thinking with Gaze: Sequential Eye-Tracking as Visual Reasoning Supervision for Medical VLMs
School of Computing, University of Georgia（佐治亚大学计算机学院）；Department of Computer Science and Engineering, University of Texas, Arlington（德克萨斯大学阿灵顿分校计算机科学与工程系）；Massachusetts General Hospital, Harvard Medical School（麻省总医院哈佛医学院）；Department of Biomedical Engineering, New Jersey Institute of Technology（新泽西理工学院生物医学工程系）
Deep Gaze I: Boosting Saliency Prediction with Feature Maps Trained on ImageNet
Werner Reichardt Centre for Integrative Neuroscience（维尔纳·赖卡特综合神经科学中心）；University Tübingen（图宾根大学）
最近的研究结果表明，国家的最先进的显着性模型在预测固定远不是最佳的。这种性能上的不足被归因于无法对诸如对象之类的高级图像特征的影响进行建模。最近将深度神经网络应用于物体识别等任务的开创性进展表明，它们能够捕获这种结构。然而，训练这些网络所需的大量训练数据使得它们难以直接应用于显著性预测。我们提出了一种新的方法，重用现有的神经网络，已预先训练的目标识别模型的固定预测的任务。使用Krizhevsky et al.（2012）的著名网络，我们提出了一个新的显着性模型，该模型在MIT显着性基准上的表现明显优于所有最先进的模型。我们表明，这个网络的结构允许新的见解，在心理物理学的固定选择和潜在的神经实现。为了训练我们的网络，我们建立在最近的工作建模的显着点过程。
From Gaze to Insight: Bridging Human Visual Attention and Vision Language Model Explanation for Weakly-Supervised Medical Image Segmentation
Department of Engineering Science, University of Oxford（工程科学系，牛津大学）；Department of Automation, Tsinghua University（自动化系，清华大学）；School of Information Science and Technology, Northwest University（信息科学与技术学院，西北大学）
From Gaze to Meaning: A Training-Free AI Agent for Unified Grounding and Explanation
Centre for Vision, Speech and Signal Processing (CVSSP), University of Surrey（萨里大学视觉、语音与信号处理中心）；Surrey Institute for People-Centred AI, University of Surrey（萨里大学以人为本人工智能研究所）
Toddlers' Active Gaze Behavior Supports Self-Supervised Object Learning
Frankfurt Institute for Advanced Studies（弗赖堡高级研究所）；Xidian-FIAS International Joint Research Center（西电-弗赖堡国际联合研究中心）；University of Texas at Austin（德克萨斯大学奥斯汀分校）
When Eyes Betray AI: Social Gaze Consistency as a Semantic Cue for AI-Generated Image Detection
School of Computer Engineering（计算机工程学院）；Hoseo University（Hoseo大学）；School of Electronic Engineering（电子工程学院）；Soongsil University（Soongsil大学）；School of Computer Science（计算机科学学院）；University of Illinois at Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）
