Self-attention has the promise of improving computer vision systems due to parameter-independent scaling of receptive fields and content-dependent interactions, in contrast to parameter-dependent scaling and content-independent interactions of convolutions. Self-attention models have recently been shown to have encouraging improvements on accuracy-parameter trade-offs compared to baseline convolutional models such as ResNet-50. In this work, we aim to develop self-attention models that can outperform not just the canonical baseline models, but even the high-performing convolutional models. We propose two extensions to self-attention that, in conjunction with a more efficient implementation of self-attention, improve the speed, memory usage, and accuracy of these models. We leverage these improvements to develop a new self-attention model family, HaloNets, which reach state-of-the-art accuracies on the parameter-limited setting of the ImageNet classification benchmark. In preliminary transfer learning experiments, we find that HaloNet models outperform much larger models and have better inference performance. On harder tasks such as object detection and instance segmentation, our simple local self-attention and convolutional hybrids show improvements over very strong baselines. These results mark another step in demonstrating the efficacy of self-attention models on settings traditionally dominated by convolutional models.
Semantic-Guided Natural Language and Visual Fusion for Cross-Modal Interaction Based on Tiny Object Detection
Department of Electrical Engineering, National Formosa University, Taiwan(台湾国立Formosa大学电子工程系);Department of Electrical Engineering, National Taipei University, Taiwan(台湾国立台北大学电子工程系);College of Artificial Intelligence and Green Energy, National Yang Ming Chiao Tung University, Taiwan(台湾国立阳明交通大学人工智能与再生能源学院)
Performance Optimization of YOLO-FEDER FusionNet for Robust Drone Detection in Visually Complex Environments
Institute for the Protection of Terrestrial Infrastructures, German Aerospace Center(地面基础设施保护研究所,德国航天中心);ACIDA Lab, University of Applied Sciences Darmstadt(达姆施塔特应用科学大学ACIDA实验室);European University of Technology(欧洲技术大学)
Efficient Remote Sensing Instance Segmentation with Linear-Time State Space Distilled Visual Foundation Models
Shen Yuan Honors College, Beihang University(北京航空航天大学沈元荣誉学院);State Key Laboratory of Virtual Reality Technology and Systems, Beihang University(北京航空航天大学虚拟现实技术与系统国家重点实验室);Qian Xuesen Laboratory of Space Technology, China Academy of Space Technology(中国空间技术研究院钱学森空间技术实验室)
MedNeXt-v2: Scaling 3D ConvNeXts for Large-Scale Supervised Representation Learning in Medical Image Segmentation
German Cancer Research Center (DKFZ) Heidelberg, Division of Medical Image Computing(德国癌症研究中心(DKFZ)海德堡,医学影像计算部);Faculty of Mathematics and Computer Science, Heidelberg University(海德堡大学数学与计算机科学学院);HIDSS4Health - Helmholtz Information and Data Science School for Health, Karlsruhe/Heidelberg, Germany(HIDSS4Health - 欧洲信息与数据科学健康学校,卡尔斯鲁厄/海德堡,德国);Medical Faculty Heidelberg, Heidelberg University(海德堡大学医学学院);National Center for Tumor Diseases (NCT), Heidelberg, Germany(肿瘤国家研究中心(NCT),海德堡,德国);Helmholtz Imaging, German Cancer Research Center, Germany(海德堡成像,德国癌症研究中心,德国);Pattern Analysis and Learning Group, Department of Radiation Oncology, Heidelberg University Hospital(模式分析与学习组,放射肿瘤学部,海德堡大学医院)
Vision Mamba: Efficient Visual Representation Learning with Bidirectional State Space Model
School of EIC, Huazhong University of Science & Technology(华中科技大学电子信息与通信学院);Institute of Artificial Intelligence, Huazhong University of Science & Technology(华中科技大学人工智能研究院);Horizon Robotics(地平线机器人);Beijing Academy of Artificial Intelligence(北京智源人工智能研究院)
Language Adaptive Weight Generation for Multi-task Visual Grounding
Zhejiang University(浙江大学);Zhejiang University-University of Illinois Urbana-Champaign Institute(浙江大学-伊利诺伊大学厄巴纳-香槟分校学院);Hikvision Research Institute(海康威视研究院);Shanghai AI Laboratory(上海人工智能实验室);Shanghai Institute for Advanced Study of Zhejiang University(浙江大学上海高等研究院)
DINOv3 Visual Representations for Blueberry Perception Toward Robotic Harvesting
Bio-Sensing, Automation, and Intelligence Laboratory, Department of Agricultural and Biological Engineering, Institute of Food and Agricultural Sciences, University of Florida(生物感知、自动化与智能实验室,农业与生物工程系,食品与农业科学研究院,佛罗里达大学);Department of Biomedical Engineering, Georgia Institute of Technology(生物医学工程系,佐治亚理工学院)
Look Before Acting: Enhancing Vision Foundation Representations for Vision-Language-Action Models
State Key Laboratory of Multimedia Information Processing, School of Computer Science, Peking University(多媒体信息处理国家重点实验室,计算机学院,北京大学);Simplexity Robotics(Simplexity机器人公司);The Chinese University of Hong Kong(香港中文大学)