Visual attention mechanisms have proven to be integrally important constituent components of many modern deep neural architectures. They provide an efficient and effective way to utilize visual information selectively, which has shown to be especially valuable in multi-modal learning tasks. However, all prior attention frameworks lack the ability to explicitly model structural dependencies among attention variables, making it difficult to predict consistent attention masks. In this paper we develop a novel structured spatial attention mechanism which is end-to-end trainable and can be integrated with any feed-forward convolutional neural network. This proposed AttentionRNN layer explicitly enforces structure over the spatial attention variables by sequentially predicting attention values in the spatial mask in a bi-directional raster-scan and inverse raster-scan order. As a result, each attention value depends not only on local image or contextual information, but also on the previously predicted attention values. Our experiments show consistent quantitative and qualitative improvements on a variety of recognition tasks and datasets; including image categorization, question answering and image generation.
Deep Attentional Structured Representation Learning for Visual Recognition
EPFL(洛桑联邦理工学院)
结构化表示,如Bags of Words,VLAD和Fisher Vectors,已被证明在处理复杂的视觉识别任务方面非常有效。因此,它们最近已被纳入深度架构中。然而,虽然有效,但由此产生的深度结构化表示学习策略通常会从整个图像中聚合局部特征,忽略了这样一个事实,即在复杂的识别任务中,某些区域提供比其他区域更多的区分信息。在本文中,我们引入了一个注意结构化的表示学习框架,它在聚合过程中引入了图像特定的注意力机制。我们的框架学习以端到端的方式联合预测图像类标签和注意力地图,并且除了目标标签之外没有任何其他监督。正如我们的实验所证明的那样,这始终优于注意力较少的结构化表示学习,并在标准场景识别和细粒度分类基准上产生最先进的结果。
Variational Structured Attention Networks for Deep Visual Representation Learning
Harbin Institute of Technology(哈尔滨工业大学);INRIA(法国国家信息与自动化研究所);Hong Kong University of Science and Technology(香港科技大学);University of Trento(特伦托大学);Fondazione Bruno Kessler(布鲁诺·凯斯勒基金会)
Learning Deep Structured Multi-Scale Features using Attention-Gated CRFs for Contour Prediction
The University of Trento(特伦托大学);The University of Sydney(悉尼大学);INRIA(法国国家信息与自动化研究所);University of Perugia(佩鲁贾大学);The Chinese University of Hong Kong(香港中文大学)
University of Amsterdam(阿姆斯特丹大学);Priberam Labs(Priberam实验室);Institute of Systems and Robotics(系统与机器人研究所);LUMLIS (Lisbon ELLIS Unit)(LUMLIS(里斯本ELLIS单元));Unbabel(Unbabel公司)
Harbin Institute of Technology, Shenzhen(哈尔滨工业大学(深圳));Great Bay University(大湾区大学);Hefei University of Technology(合肥工业大学);Macao Polytechnic University(澳门理工学院)
Beyond Classification: Structured Supervision Aligns Visual Evidence with Medical Semantics
Zhejiang University(浙江大学);The Second Affiliated Hospital Zhejiang University(浙江大学医学院附属第二医院);Transvascular Implantation Devices Research Institute and State Key Laboratory of Transvascular Implantation Devices(经血管植入器械研究院及经血管植入器械全国重点实验室);Henan Normal University(河南师范大学);University of Southampton(南安普顿大学);School of Software Technology Zhejiang University(浙江大学软件学院)
Advancing Graph Neural Networks with HL-HGAT: A Hodge-Laplacian and Attention Mechanism Approach for Heterogeneous Graph-Structured Data
National University of Singapore(新加坡国立大学);the Hong Kong Polytechnic University(香港理工大学);Fujian Agriculture and Forestry University(福建农林大学);the University of Wisconsin-Madison(威斯康星大学麦迪逊分校)
OneDrive: Unified Multi-Paradigm Driving with Vision-Language-Action Models
State Key Laboratory of Multimodal Artificial Intelligence Systems, CASIA School of Artificial Intelligence, University of Chinese Academy of Sciences AutoLab, School of Artificial Intelligence, Shanghai Jiao Tong University Voyager Research, Didi Chuxing School of Information Science(多模态人工智能系统国家重点实验室,中国科学院自动化所人工智能学院,中国科学院大学,AutoLab,上海交通大学人工智能学院,Voyager Research,滴滴出行,信息科学与技术学院)
GASL: Guided Attention for Sparsity Learning in Deep Neural Networks
Virginia Tech(弗吉尼亚理工大学);Howard University(霍华德大学);West Virginia University(西弗吉尼亚大学);Department of Computer Science(计算机科学系);Blacksburg, VA, USA(美国弗吉尼亚州布莱克斯堡);Washington DC, USA(美国华盛顿特区);Morgantown, WV, USA(美国西弗吉尼亚州摩根敦)
Interpretable Vision Transformers in Image Classification via SVDA
Dept. Electrical and Computer Engineering(电子与计算机工程系);Democritus University of Thrace(德摩克利特大学);Athena Research Center(雅典研究中心);University Campus at Kimmeria(基米里亚大学校园)
SparseSAM: Structured Sparsification of Activations in Segment Anything Models
University of Illinois at Urbana-Champaign(伊利诺伊大学厄巴纳-香槟分校);College of Engineering & Computer Science, VinUniversity(Vin大学工程与计算机科学学院);VinUni-Illinois Smart Health Center, VinUniversity(Vin大学-伊利诺伊智能健康中心);DFKI;Max Planck Research School for Intelligent Systems (IMPRS-IS)(马克斯·普朗克智能系统研究学校);University of Stuttgart(斯图加特大学)
Center for Foundation Models and Generative AI, Northwestern University(基础模型与生成式AI中心,西北大学);Department of Computer Science, Northwestern University(计算机科学系,西北大学);Ensemble AI;University of California, San Diego(加州大学圣地亚哥分校);Department of Statistics and Data Science, Northwestern University(统计与数据科学系,西北大学)