发表机构
Nanjing University of Science and Technology; State Key Laboratory of Intelligent Manufacturing of Advanced Construction Machinery; Zhejiang University; Nanjing Normal University（南京理工大学; 先进工程机械智能制造国家重点实验室; 浙江大学; 南京师范大学）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
Cut-ViT是一种基于Gram锚定子空间一致性的任务特定视觉模型剪枝方法，可高效生成高鲁棒性子网络，在6项任务9个数据集上达SOTA性能，耗时与内存开销仅为旧方法的约五分之一和近一半。
AI 中文摘要
剪枝视觉基础模型已受到广泛关注，但现有方法聚焦于单一数据集上的刚性点对点token对齐进行剪枝，存在两个局限：i）鲁棒性下降；ii）任务特定性不足。为解决这些局限，我们提出名为Cut-ViT的任务特定剪枝流水线。具体而言，我们首先从空间和语义视角构建Gram锚定矩阵，并进行子空间分解以提取对应子空间基。随后采用与基无关的残差约束，在空间和通道维度上对齐原生DINOv3模型与剪枝后模型的Gram子空间，使子网络能够继承原生DINOv3的鲁棒特征表示。此外，我们设计了频谱熵自适应方法，该方法量化特征流形在空间和通道维度的信息密度，从而使剪枝目标适配特定下游任务。实验表明，Cut-VT在单张A100 GPU上仅需约1分钟即可获得不同稀疏度水平的子网络，与先前方法相比，仅使用20.9%的时间和45.5%的GPU内存，同时在9个数据集上的6项任务中实现了SOTA性能。
英文摘要
Pruning visual foundation models has attracted considerable attention. However, existing methods focus on rigid point-to-point token alignment on a single dataset for pruning, suffering from two limitations: i) robustness degradation, and ii) task-specificity deficiency. To address these limitations, we propose a task-specific pruning pipeline, named Cut-ViT. Specifically, we first construct gram anchoring matrices from both spatial and semantic perspectives, and perform the subspace decomposition to extract the corresponding subspace bases. Basis-agnostic and residual constraints are then adopted to align the gram subspaces between the native and pruned DINOv3 models along spatial and channel dimensions, enabling subnetworks to inherit robust feature representations of native DINOv3. Furthermore, we design spectral entropy adaptation, which quantifies the information density of feature manifolds along spatial and channel dimensions, thereby adapting the pruning objective to specific downstream tasks. Experiments show that Cut-ViT requires approximately one minute on a single A100 GPU to obtain subnetworks at various sparsity levels, using only 20.9% of the time and 45.5% of the GPU memory compared with previous methods, while achieving SOTA performance on six tasks across nine datasets.
CommentsAccepted by ECCV2026
