arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 1906.08764cs.CV
收藏

深入理解深度神经网络中的人类和机器注意力

Understanding More about Human and Machine Attention in Deep Neural Networks

The Chinese University of Hong Kong(香港中文大学);Mohamed bin Zayed University of Artificial Intelligence(穆罕默德·本·扎耶德人工智能大学);South China University of Technology(华南理工大学);University of Electronic Science and Technology of China(电子科技大学);Inception Institute of Artificial Intelligence(起源人工智能研究所)

Qiuxia Lai, Salman Khan, Yongwei Nie, Jianbing Shen, Hanqiu Sun, Ling Shao

摘要:

人类视觉系统可以选择性地关注场景的部分以快速感知,这是一种被称为人类注意力的生物机制。受此启发,最近的深度学习模型对注意力机制进行编码,将注意力集中在输入信号中与任务最相关的部分进行进一步处理,这被称为机器/神经/人工注意力。理解人类和机器注意力之间的关系对于解释和设计神经网络非常重要。许多著作声称,注意力机制通过解释神经网络看起来的位置,提供了一个额外的维度。然而,最近的研究表明,人工注意力地图并不总是与常见的直觉相一致。鉴于这些相互矛盾的证据,本文对人工注意力和人类注意力在神经网络设计中的应用进行了系统的研究。通过三个计算机视觉任务的实例、不同的代表性骨干和著名的体系结构、相应的真实人类注视数据,并系统地进行大规模的定量研究,我们量化了人工注意力和人类视觉注意力之间的一致性,并通过初步回答与人类和人工注意力机制相关的几个关键问题,为现有的人工注意力机制提供了新的见解。总体结果表明,人类的注意力可以基准有意义的“地面真相”在注意力驱动的任务,人工注意力越接近人类的注意力,性能越好;对于更高层次的视觉任务,这是个案。对于注意力驱动的任务来说,明确地强制人工和人类注意力之间更好的对齐以提高性能是可取的;这种对齐也将提高更高级别计算机视觉任务的网络可解释性。

英文摘要:

Human visual system can selectively attend to parts of a scene for quick perception, a biological mechanism known as Human attention. Inspired by this, recent deep learning models encode attention mechanisms to focus on the most task-relevant parts of the input signal for further processing, which is called Machine/Neural/Artificial attention. Understanding the relation between human and machine attention is important for interpreting and designing neural networks. Many works claim that the attention mechanism offers an extra dimension of interpretability by explaining where the neural networks look. However, recent studies demonstrate that artificial attention maps do not always coincide with common intuition. In view of these conflicting evidence, here we make a systematic study on using artificial attention and human attention in neural network design. With three example computer vision tasks, diverse representative backbones, and famous architectures, corresponding real human gaze data, and systematically conducted large-scale quantitative studies, we quantify the consistency between artificial attention and human visual attention and offer novel insights into existing artificial attention mechanisms by giving preliminary answers to several key questions related to human and artificial attention mechanisms. Overall results demonstrate that human attention can benchmark the meaningful `ground-truth' in attention-driven tasks, where the more the artificial attention is close to human attention, the better the performance; for higher-level vision tasks, it is case-by-case. It would be advisable for attention-driven tasks to explicitly force a better alignment between artificial and human attention to boost the performance; such alignment would also improve the network explainability for higher-level computer vision tasks.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑