Transformers increasingly dominate the machine learning landscape across many tasks and domains, which increases the importance for understanding their outputs. While their attention modules provide partial insight into their inner workings, the attention scores have been shown to be insufficient for explaining the models as a whole. To address this, we propose B-cos transformers, which inherently provide holistic explanations for their decisions. Specifically, we formulate each model component - such as the multi-layer perceptrons, attention layers, and the tokenisation module - to be dynamic linear, which allows us to faithfully summarise the entire transformer via a single linear transform. We apply our proposed design to Vision Transformers (ViTs) and show that the resulting models, dubbed Bcos-ViTs, are highly interpretable and perform competitively to baseline ViTs on ImageNet. Code will be made available soon.
Transformer-Based Explainable Deep Learning for Breast Cancer Detection in Mammography: The MammoFormer Framework
Morgan State University（摩根州立大学）；Wrexham University（雷克瑟姆大学）；Federal University of Technology, Minna（明纳联邦理工大学）；Federal University of Education, Kano（卡诺联邦教育大学）
Keypoint Counting Classifiers: Turning Vision Transformers into Self-Explainable Models Without Training
Department of Physics and Technology（物理与技术系）；UiT The Arctic University of Norway（北摩尔曼斯克大学）；Department of Applied Mathematics and Computer Science（应用数学与计算机科学系）；Technical University of Denmark（技术大学）
Swin Deformable Attention U-Net Transformer (SDAUT) for Explainable Fast MRI
National Heart and Lung Institute, Imperial College London（帝国理工学院国家心肺研究所）；Cardiovascular Research Centre, Royal Brompton Hospital（皇家布朗普顿医院心血管研究中心）；School of Biomedical Engineering, Sun Yat-sen University（中山大学生物医学工程学院）
An Explainable Transformer Model for Alzheimer's Disease Detection Using Retinal Imaging
Nanotechnology, Biotechnology, Information Technology and Cognitive Science Laboratory, University of Tehran（特拉布宗大学纳米技术、生物技术、信息技术和认知科学实验室）；Department of Mechatronics, School of Intelligent Systems, College of Interdisciplinary Science and Technology, University of Tehran（特拉布宗大学机电系，智能系统学院，跨学科科学与技术学院）；School of Science and Technology, Faculty of Science, Agriculture, Business and Law, University of New England（新英格兰大学科学与技术学院，科学、农业、商业与法律学院）；Graduate School of Health, University of Technology Sydney（悉尼技术大学健康研究生院）；School of Medicine and Public Health, College of Health, Medicine and Wellbeing, The University of Newcastle（新castle大学医学与公共卫生学院，健康、医学与福祉学院）
TinyViT-Batten: Few-Shot Vision Transformer with Explainable Attention for Early Batten-Disease Detection on Pediatric MRI
Department of Applied Medical Sciences（应用医学科学系）；RareGen Youth Network 501(c)(3)（RareGen青年网络501(c)(3)）；Harvard University（哈佛大学）；Carnegie Mellon University（卡内基梅隆大学）；Mellon College of Science（梅隆科学学院）
A Graph-Augmented knowledge Distillation based Dual-Stream Vision Transformer with Region-Aware Attention for Gastrointestinal Disease Classification with Explainable AI
Department of Computer Science and Engineering（计算机科学与工程系）
Transformer-Driven Multimodal Fusion for Explainable Suspiciousness Estimation in Visual Surveillance
Big Data Research and Supercomputing Division, CSIR Fourth Paradigm Institute（CSIR第四范式研究所大数据研究与超级计算部门）；Department of Electrical Engineering, Bharti School of Telecommunication, Yardi School of Artificial Intelligence, IIT Delhi（电信学院电子工程系、Yardi人工智能学院、德里理工学院）
Automatic Diagnosis of Myocarditis Disease in Cardiac MRI Modality using Deep Transformers and Explainable Artificial Intelligence
The Graduate School of Biomedical Engineering, UNSW Sydney（新南威尔士大学悉尼分校生物医学工程研究生院）；Data Science and Computational Intelligence Institute, University of Granada（格拉纳达大学数据科学与计算智能研究所）；University of La Rioja（拉里奥哈大学）；University of Technology Sydney (UTS)（悉尼科技大学）；Macquarie University（麦考瑞大学）；University of Leicester（莱斯特大学）；Deakin University（迪肯大学）；University of Cambridge（剑桥大学）；University of Southern Queensland（南昆士兰大学）；The University of New South Wales（新南威尔士大学）；AI-enabled Processes (AIP) Research Centre, Macquarie University（麦考瑞大学人工智能赋能流程研究中心）
An explainable vision transformer with transfer learning based efficient drought stress identification
Dept. of Computer Science & Engineering（计算机科学与工程系）；North Eastern Regional Institute of Science and Technology（北东地区科学与技术研究所）；Dept. of Bio-Science and Bio-Engineering（生物科学与生物工程系）；Indian Institute of Technology Guwahati（印度理工学院古瓦哈蒂分校）；Dept. of Physics（物理系）
Deciphering Heartbeat Signatures: A Vision Transformer Approach to Explainable Atrial Fibrillation Detection from ECG Signals
Walkky（沃基公司）；Chobanian & Avedisian School of Medicine, Boston University（波士顿大学乔巴尼亚与阿夫迪西安医学院）；Clemson University（克莱姆森大学）；Northwell Health Physician Partners（诺斯韦尔健康医师伙伴）；Lenox Hill Hospital（莱诺克斯山医院）
AttnLRP: Attention-Aware Layer-Wise Relevance Propagation for Transformers
Fraunhofer Heinrich-Hertz-Institute（弗劳恩霍夫海因茨·赫兹研究所）；Technische Universität Berlin（柏林工业大学）；BIFOLD – Berlin Institute for the Foundations of Learning and Data（柏林学习与数据基础研究所（BIFOLD））
Joint rotational invariance and adversarial training of a dual-stream Transformer yields state of the art Brain-Score for Area V4
The Center for Brains, Minds and Machines（大脑、心智与机器中心）；Massachusetts Institute of Technology（麻省理工学院）
现代在大脑评分竞赛中获得高分的视觉模型并不源于Vision Transformers。然而，在本文中，我们通过展示双流Transformer，CrossViT $~\textit{a la}$ Chen et al.（2021），在联合旋转不变和对抗优化程序下，在2022年总Brain-Score竞赛中获得第二名，提供了反对Vision Transformers（ViT）在感知上与人类视觉表示不一致的意外趋势的证据（Schrimpf et al.，2020 b）在所有视觉类别中平均，并且在比赛时在V4区域的最高可解释方差中排名第一。此外，我们目前的基于变换器的模型还实现了区域V4、IT和行为的更大的可解释方差，而不是生物启发的CNN（ResNet 50），其集成了类似于额叶V1的计算模块（Dapello等人，2020年）。为了评估优化方案对CrossViT架构的贡献，我们对不同优化的CrossViT的对抗鲁棒性，常见的腐败基准，中腹部刺激解释和特征反转进行了几次额外的实验。与我们最初的预期，我们的家庭的结果提供了临时支持$\textit{"All roads lead to Rome"}$参数通过联合优化规则，即使是非生物学动机的模型，如Vision Transformers的视觉。代码可在https://github.com/williamberrios/BrainScore-T…
From ACR O-RADS 2022 to Explainable Deep Learning: Comparative Performance of Expert Radiologists, Convolutional Neural Networks, Vision Transformers, and Fusion Models in Ovarian Masses
Danube Private University（多瑙河私立大学）；Urmia University of Medical Science（乌尔米耶医科大学）；Universiti Malaya（马来亚大学）；Universiti Teknologi MARA Selangor（玛拉工艺大学雪兰莪分校）；University of Southern Queensland（南昆士兰大学）；Austrian Center for Medical Innovation and Technology（奥地利医学创新与技术中心）；Medical University of Vienna（维也纳医科大学）
Explainable Convolutional Neural Networks for Retinal Fundus Classification and Cutting-Edge Segmentation Models for Retinal Blood Vessels from Fundus Images
Ahsanullah University of Science and Technology（阿萨努拉科技大学）
Redundancy-aware Transformer for Video Question Answering
National University of Singapore（新加坡国立大学）；University of Science and Technology of China（中国科学技术大学）；Institute of Dataspace, Hefei Comprehensive National Science Center（合肥综合性国家科学中心数据空间研究院）
XMACNet: An Explainable Lightweight Attention based CNN with Multi Modal Fusion for Chili Disease Classification
Vellore Institute of Technology（韦洛尔理工学院）；Angel College of Engineering and Technology（安吉尔工程技术学院）；Vel Tech Rangarajan Dr.Sagunthala R&D Institute of Science and Technology（Vel Tech朗加拉詹·萨贡塔拉博士科技研究院）；Bannari Amman Institute of Technology（班纳里·安曼理工学院）
