arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2603.09613cs.CV
收藏

基于眼动的图像分类方法:使用视觉Transformer注意力图

A Saccade-inspired Approach to Image Classification using Vision Transformer Attention Maps

Institut de Neurosciences de la Timone, Aix-Marseille Université, CNRS(神经科学蒂姆研究所,艾克斯-马赛大学,CNRS);Laboratoire d’Électronique, Antennes et Télécommunications, Université Côte d’Azur, CNRS(电子、天线和电信实验室,蔚蓝海岸大学,CNRS)

Matthis Dallain, Laurent Rodriguez, Laurent Udo Perrinet, Benoît Miramond

AI总结:

本文提出了一种受眼动启发的图像分类方法,利用DINO生成的注意力图,通过模拟眼动过程提高分类性能。

摘要:

人类视觉在严格代谢限制下仍能实现卓越的感知性能。关键因素是选择性注意机制,由快速的眼球运动驱动,不断将高分辨率的视网膜黄斑重新定位到任务相关的位置,不同于传统AI系统对整个图像等量处理。我们的工作旨在从人类视觉系统中汲取灵感,创造更智能、更高效的图像处理模型。使用DINO,一种自监督的视觉Transformer,它产生与人类目光模式惊人相似的注意力图,我们探索了一种受眼动启发的方法,以在视觉空间中聚焦关键区域的信息处理。为此,我们使用ImageNet数据集在标准分类任务中,并测量每次连续眼动如何影响模型的分类得分。这种选择性处理策略保留了大部分完整图像分类性能,并在某些情况下甚至可以超越它。通过与用于人类目光预测的现有显著性模型进行基准测试,我们证明DINO在选择信息区域方面提供了更优的固定指导。这些发现突显了视觉Transformer注意力作为生物启发式主动视觉的有前途的基础,并为高效、神经形态视觉处理开辟了新方向。

英文摘要:

Human vision achieves remarkable perceptual performance while operating under strict metabolic constraints. A key ingredient is the selective attention mechanism, driven by rapid saccadic eye movements that constantly reposition the high-resolution fovea onto task-relevant locations, unlike conventional AI systems that process entire images with equal emphasis. Our work aims to draw inspiration from the human visual system to create smarter, more efficient image processing models. Using DINO, a self-supervised Vision Transformer that produces attention maps strikingly similar to human gaze patterns, we explore a saccade inspired method to focus the processing of information on key regions in visual space. To do so, we use the ImageNet dataset in a standard classification task and measure how each successive saccade affects the model's class scores. This selective-processing strategy preserves most of the full-image classification performance and can even outperform it in certain cases. By benchmarking against established saliency models built for human gaze prediction, we demonstrate that DINO provides superior fixation guidance for selecting informative regions. These findings highlight Vision Transformer attention as a promising basis for biologically inspired active vision and open new directions for efficient, neuromorphic visual processing.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑