Foundational Vision Transformers (ViTs) have limited effectiveness in tasks requiring fine-grained spatial understanding, due to their fixed pre-training resolution and inherently coarse patch-level representations. These challenges are especially pronounced in dense prediction scenarios, such as open-vocabulary segmentation with ViT-based vision-language models, where high-resolution inputs are essential for accurate pixel-level reasoning. Existing approaches typically process large-resolution images using a sliding-window strategy at the pre-training resolution. While this improves accuracy through finer strides, it comes at a significant computational cost. We introduce SPAR: Single-Pass Any-Resolution ViT, a resolution-agnostic dense feature extractor designed for efficient high-resolution inference. We distill the spatial reasoning capabilities of a finely-strided, sliding-window teacher into a single-pass student using a feature regression loss, without requiring architectural changes or pixel-level supervision. Applied to open-vocabulary segmentation, SPAR improves single-pass baselines by up to 10.5 mIoU and even surpasses the teacher, demonstrating effectiveness in efficient, high-resolution reasoning. Code: https://github.com/naomikombol/SPAR
CLIPSelf: Vision Transformer Distills Itself for Open-Vocabulary Dense Prediction
Nanyang Technological University（南洋理工大学）；The Chinese University of Hong Kong（香港中文大学）；The University of Hong Kong（香港大学）；SenseTime Research and Tetras.AI（商汤科技研究院与Tetras.AI）；Shanghai AI Laboratory（上海人工智能实验室）
针对 CLIP ViT 从全局图像迁移到局部区域时的域偏移问题，本文提出 CLIPSelf 自蒸馏方法，在无需区域-文本对的情况下增强区域-语言对齐，并在多项开放词汇密集预测基准上达到最优性能。
Plug-in Feedback Self-adaptive Attention in CLIP for Training-free Open-Vocabulary Segmentation
University of Toronto（多伦多大学）；China Agricultural University（中国农业大学）；Concordia University（Concordia大学）；McMaster University（麦马斯特大学）；Beijing Jiaotong University（北京交通大学）
Dynamic-Robust Photometric-Semantic Reconstruction for Open-Vocabulary 3D Scene Understanding
State Key Laboratory of Multimodal Artificial Intelligence Systems (MAIS)（多模态人工智能系统国家重点实验室）；Institute of Automation, Chinese Academy of Sciences (CASIA)（中国科学院自动化研究所）；School of Information Science and Technology, ShanghaiTech University（上海科技大学信息科学与技术学院）；The Chinese University of Hong Kong（香港中文大学）；Deepeleph Intelligent Technology（深智科技）
Exploring Multi-Modal Contextual Knowledge for Open-Vocabulary Object Detection
Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；University of the Chinese Academy of Sciences（中国科学院大学）；Peng Cheng Laboratory（鹏城实验室）；Tencent Youtu Lab（腾讯优图实验室）
Scaling object taxonomies is one of the important steps toward a robust real-world deployment of recognition systems. We have faced remarkable progress in images since the introduction of the LVIS benchmark. To continue this success in videos, a new video benchmark, TAO, was recently presented. Given the recent encouraging results from both detection and tracking communities, we are interested in marrying those two advances and building a strong large vocabulary video tracker. However, supervisions in LVIS and TAO are inherently sparse or even missing, posing two new challenges for training th…
OV-COAST: Cost Aggregation with Optimal Transport for Open-Vocabulary Semantic Segmentation
Kotak IISc AI-ML Centre, Indian Institute of Science, Bengaluru, India（Kotak IISc AI-ML 中心，印度科学学院，班加罗尔，印度）；Robert Bosch Centre for Cyber Physical Systems, Indian Institute of Science, Bengaluru, India（罗伯特·博世网络物理系统中心，印度科学学院，班加罗尔，印度）；Department of Aerospace Engineering, Indian Institute of Science, Bengaluru, India（航空工程系，印度科学学院，班加罗尔，印度）
SAM-MI: A Mask-Injected Framework for Enhancing Open-Vocabulary Semantic Segmentation with SAM
State Key Laboratory of Multimodal Artificial Intelligence Systems (MAIS)（多模态人工智能系统国家重点实验室）；Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）
Revisiting Token Compression for Accelerating ViT-based Sparse Multi-View 3D Object Detectors
PCA Lab, School of Computer Science and Engineering, Nanjing University of Science and Technology（计算机科学与工程学院，南京理工大学PCA实验室）；PCA Lab, School of Intelligence Science and Technology, Nanjing University（智能科学与技术学院，南京大学PCA实验室）
Bridging 3D Gaussians and Semantic Occupancy for Comprehensive Open-Vocabulary Scene Understanding from Unposed Images
The Hong Kong Polytechnic University（香港理工大学）；Eastern Institute of Technology（东方理工大学）；Shanghai Jiao Tong University（上海交通大学）；Wuhan University（武汉大学）；Great Wall Motor（长城汽车）；Tsinghua University（清华大学）；Ningbo Institute of Digital Twin, Eastern Institute of Technology（东方理工大学宁波数字孪生研究院）
OpenM3D: Open Vocabulary Multi-view Indoor 3D Object Detection without Human Annotations
National Tsing Hua University（国立清华大学）；Amazon（亚马逊）；Cornell University（康奈尔大学）；Carnegie Mellon University（卡内基梅隆大学）；National Yang Ming Chiao Tung University（国立阳明交通大学）
Confidence Scores in Open-Vocabulary Detection Are a Biased Mixture of Scale and Semantics
Institute of Intelligence Systems, National Yang Ming Chiao Tung University（国立阳明交通大学智能系统研究所）；College of Artificial Intelligence and Green Energy, National Yang Ming Chiao Tung University（国立阳明交通大学人工智能与绿色能源学院）
CL-CLIP: CLIP-Based Continual Learning Framework with Cost-Volume Category Decoupling for Object Detection
National College for Excellent Engineers, Beihang University（卓越工程师学院，北京航空航天大学）；AI Research, Qihoo 360（360人工智能研究院，奇虎360）；School of Electronic Information Engineering, Beihang University（电子信息学院，北京航空航天大学）；School of Cyber Science and Technology, Beihang University（网络安全科学与技术学院，北京航空航天大学）；School of Computer Science and Engineering, Beihang University（计算机科学与工程学院，北京航空航天大学）；State Key Laboratory of Media Convergence and Communication, Communication University of China（媒体融合与传播国家重点实验室，中国传媒大学）；Institute of Information Technology, Mathematics and Mechanics, Lobachebsky University（信息技术、数学与力学学院，洛瓦茨基大学）；School of Artificial Intelligence, Beihang University（人工智能学院，北京航空航天大学）
OVER-NAV: Elevating Iterative Vision-and-Language Navigation with Open-Vocabulary Detection and StructurEd Representation
The University of Hong Kong（香港大学）；Sun Yat-sen University（中山大学）；Digital Content Technology Center, Tencent Games（腾讯游戏数字内容技术中心）；GuangDong Province Key Laboratory of Information Security Technology（广东省信息安全重点实验室）
University of Texas at Austin（德克萨斯大学奥斯汀分校）；Eindhoven University of Technology（埃因霍温理工大学）；University of Jyväskylä（于韦斯屈莱大学）；University of Twente（特文特大学）；University of Luxembourg（卢森堡大学）
Transformers have quickly shined in the computer vision world since the emergence of Vision Transformers (ViTs). The dominant role of convolutional neural networks (CNNs) seems to be challenged by increasingly effective transformer-based models. Very recently, a couple of advanced convolutional models strike back with large kernels motivated by the local-window attention mechanism, showing appealing performance and efficiency. While one of them, i.e. RepLKNet, impressively manages to scale the kernel size to 31x31 with improved performance, the performance starts to saturate as the kernel size…
DART: An Automated End-to-End Object Detection Pipeline with Data Diversification, Open-Vocabulary Bounding Box Annotation, Pseudo-Label Review, and Model Training
National University of Singapore（新加坡国立大学）；Alibaba Group（阿里巴巴集团）；Amazon（亚马逊）
The quadratic computational complexity to the number of tokens limits the practical applications of Vision Transformers (ViTs). Several works propose to prune redundant tokens to achieve efficient ViTs. However, these methods generally suffer from (i) dramatic accuracy drops, (ii) application difficulty in the local vision transformer, and (iii) non-general-purpose networks for downstream tasks. In this work, we propose a novel Semantic Token ViT (STViT), for efficient global and local vision transformers, which can also be revised to serve as backbone for downstream tasks. The semantic tokens…
Vision transformer (ViT) has achieved competitive accuracy on a variety of computer vision applications, but its computational cost impedes the deployment on resource-limited mobile devices. We explore the sparsity in ViT and observe that informative patches and heads are sufficient for accurate image recognition. In this paper, we propose a cascade pruning framework named CP-ViT by predicting sparsity in ViT models progressively and dynamically to reduce computational redundancy while minimizing the accuracy loss. Specifically, we define the cumulative score to reserve the informative patches…
Rice University（莱斯大学）；Georgia Institute of Technology（佐治亚理工学院）；Nanjing University（南京大学）
Vision Transformer (ViT) has emerged as a competitive alternative to convolutional neural networks for various computer vision applications. Specifically, ViT multi-head attention layers make it possible to embed information globally across the overall image. Nevertheless, computing and storing such attention matrices incurs a quadratic cost dependency on the number of patches, limiting its achievable efficiency and scalability and prohibiting more extensive real-world ViT applications on resource-constrained devices. Sparse attention has been shown to be a promising direction for improving ha…
