This paper introduces a novel attention mechanism, called dual attention, which is both efficient and effective. The dual attention mechanism consists of two parallel components: local attention generated by Convolutional Neural Networks (CNNs) and long-range attention generated by Vision Transformers (ViTs). To address the high computational complexity and memory footprint of vanilla Multi-Head Self-Attention (MHSA), we introduce a novel Multi-Head Partition-wise Attention (MHPA) mechanism. The partition-wise attention approach models both intra-partition and inter-partition attention simultaneously. Building on the dual attention block and partition-wise attention mechanism, we present a hierarchical vision backbone called DualFormer. We evaluate the effectiveness of our model on several computer vision tasks, including image classification on ImageNet, object detection on COCO, and semantic segmentation on Cityscapes. Specifically, the proposed DualFormer-XS achieves 81.5\% top-1 accuracy on ImageNet, outperforming the recent state-of-the-art MPViT-XS by 0.6\% top-1 accuracy with much higher throughput.
DS-TransUNet:Dual Swin Transformer U-Net for Medical Image Segmentation
Harbin Institute of Technology, Shenzhen(哈尔滨工业大学(深圳));Shenzhen Medical Biometrics Perception and Analysis Engineering Laboratory(深圳市医学生物特征感知与分析工程实验室);Bio-Computing Research Center(生物计算研究中心);Shenzhen Key Laboratory of Visual Object Detection and Recognition(深圳市视觉目标检测与识别重点实验室)
Dual-stream Transformer-GCN Model with Contextualized Representations Learning for Monocular 3D Human Pose Estimation
Northeastern University(东北大学);College of Sciences, Northeastern University(东北大学理学院);Key Laboratory of Differential Equations and Their Applications, Northeastern University(东北大学微分方程及应用重点实验室);Liaoning Provincial Department of Education(辽宁省教育厅);King’s College London(伦敦国王学院);Shenyang sport University(沈阳体育学院)
DualX-VSR: Dual Axial Spatial$\times$Temporal Transformer for Real-World Video Super-Resolution without Motion Compensation
University of Science and Technology of China(中国科学技术大学);Shanghai Artificial Intelligence Laboratory(上海人工智能实验室);Shanghai Jiao Tong University(上海交通大学);Tsinghua University(清华大学);Sun Yat-sen University(中山大学);Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences(中国科学院深圳先进技术研究院)
Efficient Partitioning Vision Transformer on Edge Devices for Distributed Inference
School of Computing, National University of Singapore(国家大学学院计算机科学学院);Information and Artificial Intelligence Institute, Zhejiang University of Finance & Economics Dongfang College(浙江财经大学东方学院信息与人工智能研究院);Department of Computer Science, ETH Zurich(苏黎世联邦理工学院计算机科学系);College of Computer Science and Technology, Zhejiang University(浙江大学计算机科学与技术学院);Lumia Ltd.(Lumia有限公司);Institute of Data Science, National University of Singapore(国家大学数据科学研究所)
Rethinking Attention Gated with Hybrid Dual Pyramid Transformer-CNN for Generalized Segmentation in Medical Imaging
Junia(朱尼亚学院);Centrale Lille(里尔中央理工学院);University of the Basque Country UPV/EHU(巴斯克大学 UPV/EHU);Université Polytechnique Hauts-de-France(上法兰西理工大学);Université de Lille(里尔大学);Ho Chi Minh City Open University(胡志明市开放大学)
TransEditor: Transformer-Based Dual-Space GAN for Highly Controllable Facial Editing
Shanghai AI Laboratory(上海人工智能实验室);Nanyang Technological University(南洋理工大学);Monash University(莫纳什大学);SenseTime Research(商汤科技研究院);Hong Kong University of Science and Technology(香港科技大学)
DualDiT: A Conditional Dual-Output Diffusion Transformer for Joint OCT Image and Segmentation Mask Generation
Instituto Universitario de Investigación en Tecnología Centrada en el Ser Humano (HUMAN-tech), Universitat Politècnica de València(瓦伦西亚理工大学以人类为中心技术大学研究所);Artikode Intelligence S.L(Artikode智能公司);Biomedical Technology Center of the Medical Faculty, University of Muenster(明斯特大学医学院生物医学技术中心);Department of Ophthalmology, University of Muenster Medical Centre(明斯特大学医学中心眼科)
GRIT: Faster and Better Image captioning Transformer Using Dual Visual Features
Tohoku University(东北大学);RIKEN Center for AIP(理化学研究所先进智能项目中心)
当前最先进的图像描述方法采用基于区域的特征,因为它们提供了描述图像内容所必需的对象级信息;它们通常由诸如Faster R-CNN之类的对象检测器提取。然而,它们有几个问题,如缺乏上下文信息,不准确检测的风险,以及高计算成本。前两个可以通过额外使用基于网格的特征来解决。然而,如何提取和融合这两种类型的功能是未知的。本文提出了一种仅基于Transformer的神经架构,称为GRIT(Grid- and Region-based图像描述Transformer),它有效地利用了这两个视觉特征来生成更好的字幕。GRIT用基于DETR的检测器取代了以前方法中使用的基于CNN的检测器,使其计算速度更快。此外,它的单片设计仅由Transformers组成,可以实现模型的端到端训练。这种创新的设计和双重视觉功能的整合带来了显着的性能改善。在多个图像描述基准上的实验结果表明,GRIT在推理精度和速度上都优于以往的方法。
Joint rotational invariance and adversarial training of a dual-stream Transformer yields state of the art Brain-Score for Area V4
The Center for Brains, Minds and Machines(大脑、心智与机器中心);Massachusetts Institute of Technology(麻省理工学院)
现代在大脑评分竞赛中获得高分的视觉模型并不源于Vision Transformers。然而,在本文中,我们通过展示双流Transformer,CrossViT $~\textit{a la}$ Chen et al.(2021),在联合旋转不变和对抗优化程序下,在2022年总Brain-Score竞赛中获得第二名,提供了反对Vision Transformers(ViT)在感知上与人类视觉表示不一致的意外趋势的证据(Schrimpf et al.,2020 b)在所有视觉类别中平均,并且在比赛时在V4区域的最高可解释方差中排名第一。此外,我们目前的基于变换器的模型还实现了区域V4、IT和行为的更大的可解释方差,而不是生物启发的CNN(ResNet 50),其集成了类似于额叶V1的计算模块(Dapello等人,2020年)。为了评估优化方案对CrossViT架构的贡献,我们对不同优化的CrossViT的对抗鲁棒性,常见的腐败基准,中腹部刺激解释和特征反转进行了几次额外的实验。与我们最初的预期,我们的家庭的结果提供了临时支持$\textit{"All roads lead to Rome"}$参数通过联合优化规则,即使是非生物学动机的模型,如Vision Transformers的视觉。代码可在https://github.com/williamberrios/BrainScore-T…