近期基于激光雷达的高性能时序3D检测器越来越多地采用基于区域的范式。它们首先生成粗略的提议,随后对区域特征进行编码与融合。然而,无差别的采样与融合往往忽略了各个点不同的贡献,并随着输入帧数的增加导致复杂度呈指数级增长。此外,任意的结构级拼接限制了全局信息的提取。在本文中,我们提出了一种焦点Token获取与缩放Transformer(FASTer),它以自适应且轻量级的方式动态选择焦点Token并精简Token序列。为强调各个Token的贡献,我们提出了一种简单但有效的自适应缩放机制,以在筛选焦点点的同时捕获几何上下文。自适应地仅存储和处理历史帧中的焦点点,极大地降低了整体复杂度。此外,提出了一种新颖的分组分层融合策略,逐步执行序列缩放和组内融合操作,以促进全局时空信息的交换。在Waymo Open Dataset上的实验表明,我们的FASTer在性能和效率上均显著优于其他最先进的检测器,同时展现出更好的灵活性和鲁棒性。代码可在 https://github.com/MSunDYY/FASTer.git 获取。
英文摘要:
Recent top-performing temporal 3D detectors based on Lidars have increasingly adopted region-based paradigms. They first generate coarse proposals, followed by encoding and fusing regional features. However, indiscriminate sampling and fusion often overlook the varying contributions of individual points and lead to exponentially increased complexity as the number of input frames grows. Moreover, arbitrary result-level concatenation limits the global information extraction. In this paper, we propose a Focal Token Acquring-and-Scaling Transformer (FASTer), which dynamically selects focal tokens and condenses token sequences in an adaptive and lightweight manner. Emphasizing the contribution of individual tokens, we propose a simple but effective Adaptive Scaling mechanism to capture geometric contexts while sifting out focal points. Adaptively storing and processing only focal points in historical frames dramatically reduces the overall complexity. Furthermore, a novel Grouped Hierarchical Fusion strategy is proposed, progressively performing sequence scaling and Intra-Group Fusion operations to facilitate the exchange of global spatial and temporal information. Experiments on the Waymo Open Dataset demonstrate that our FASTer significantly outperforms other state-of-the-art detectors in both performance and efficiency while also exhibiting improved flexibility and robustness. The code is available at https://github.com/MSunDYY/FASTer.git.
Microsoft Research(微软研究院);Microsoft Cloud + AI(微软云计算与人工智能部门)
我们提出了焦点调制网络(简称FocalNets),其中自我注意力(SA)完全被焦点调制机制所取代,用于对视觉中的令牌交互进行建模。焦点调制包括三个组件:(i)使用深度卷积层的堆栈实现的分层上下文化,以从短到长范围编码视觉上下文,(ii)门控聚合,以基于其内容选择性地收集每个查询令牌的上下文,以及(iii)元素调制或仿射变换,以将聚合的上下文注入查询。广泛的实验表明,FocalNets的性能优于最先进的SA对应物(例如,Swin和Focal Transformers)在图像分类、目标检测和分割任务上具有相似的计算成本。具体来说,具有微小和基本尺寸的FocalNets在ImageNet-1 K上实现了82.3%和83.9%的top-1准确率。在ImageNet-22 K上以224分辨率进行预训练后,当以224和384分辨率进行微调时,它分别达到了86.5%和87.3%的top-1准确率。当转移到下游任务时,FocalNets表现出明显的优势。对于使用Mask R-CNN的目标检测,使用1个\times训练的FocalNet基础比Swin对手高出2.1个点,并且已经超过了使用3个\times时间表训练的Swin(49.0 vs. 48.5)。对于使用UPerNet的语义分割,FocalNet在单尺度下的性能优于Swin 2.4,在多尺度下优于Swin(50.5 vs. 49.7)。使用大…
IFSeg: Image-free Semantic Segmentation via Vision-Language Model
Korea Advanced Institute of Science and Technology (KAIST)(韩国科学技术院);Google Research(谷歌研究院);Mohamed bin Zayed University of Artificial Intelligence (MBZUAI)(穆罕默德·本·扎耶德人工智能大学)
HALC: Object Hallucination Reduction via Adaptive Focal-Contrast Decoding
University of Chicago(芝加哥大学);Massachusetts Institute of Technology(麻省理工学院);UNC-Chapel Hill(北卡罗来纳大学教堂山分校);University of Illinois at Urbana-Champaign(伊利诺伊大学厄巴纳-香槟分校);Toyota Technological Institute at Chicago(芝加哥丰田理工学院)
FViT: A Focal Vision Transformer with Gabor Filter
College of Artificial Intelligence(人工智能学院);School of Artificial Intelligence(人工智能学院);The Key Laboratory of Intelligent Robotics of Tianjin(天津市智能机器人重点实验室)
Hong Kong University of Science and Technology(香港科技大学);University of Westminster(威斯敏斯特大学);University of Engineering & Technology Lahore(拉合尔工程技术大学);Shanghai Jiao Tong University(上海交通大学)
NeuroDETR: Multiscale Discrete Spatiotemporal Tokenization for Asynchronous EEG Event Detection
Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所);University of Chinese Academy of Sciences(中国科学院大学);Zhongguancun Academy(中关村学院);Wuhan University(武汉大学)
VisualAD: Language-Free Zero-Shot Anomaly Detection via Vision Transformer
State Key Laboratory of Opto-Electronic Information Acquisition and Protection Technology, Anhui University, Hefei, China(光电信息采集与防护技术国家重点实验室,安徽大学,合肥,中国);School of Artificial Intelligence, Anhui University, Hefei, China(人工智能学院,安徽大学,合肥,中国);College of Intelligence Science and Technology, National University of Defense Technology, Changsha, China(智能科学与技术学院,国防科技大学,长沙,中国)