arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2604.16745cs.AIcs.CV
收藏

为何无训练标记缩减崩溃:成对评分信号的内在不稳定性

Why Training-Free Token Reduction Collapses: The Inherent Instability of Pairwise Scoring Signals

Yang Shanglin

AI总结:

本文分析了无训练标记缩减方法在高压缩下的崩溃原因,提出两种诊断工具,揭示了层间缩减的固有误差放大效应及对成对相似性信号的依赖,通过CATIS验证,实现高压缩下的高精度保持。

摘要:

无训练标记缩减方法(如ToMe、ToFu、PiToMe和MCTF)采用不同的评分机制,但它们在高压缩时共享一个陡峭的崩溃现象。本文解释了这一现象。我们开发了一个诊断框架,包含两个工具:排名一致性ρ_s和非对角线相关性ρ_off,将崩溃分解为(1)层间缩减固有的信号无关误差放大效应,预测凸Pareto曲线和r_crit ∝ 1/L;以及(2)对成对相似性信号的共同依赖,其排名一致性从ρ_s=0.88降至0.27。成对排名本质上不稳定(O(N_p^2)联合扰动),而单值信号更具稳定性(O(N_p)扰动,CLT)。基于此诊断,我们提出三个设计原则,构建CATIS进行验证:单值信号提高触发阈值,三审抑制增益。在ViT-Large在63% FLOPs压缩下,CATIS保持96.9%的原始精度(81.0%)在ImageNet-1K上,而所有基线方法崩溃至43-65%。

英文摘要:

Training-free token reduction methods for Vision Transformers (ToMe, ToFu, PiToMe, and MCTF) employ different scoring mechanisms, yet they share a closely matched cliff-like collapse at high compression. This paper explains \emph{why}. We develop a diagnostic framework with two tools, ranking consistency $ρ_s$ and off-diagonal correlation $ρ_\text{off}$, that decomposes the collapse into (1)a signal-agnostic error amplifier inherent to layer-wise reduction, predicting convex Pareto curves and $r_{\text{crit}} \propto 1/L$; and (2)shared reliance on \emph{pairwise} similarity signals whose ranking consistency degrades from $ρ_s{=}0.88$ to $0.27$ in deep layers. Pairwise rankings are inherently unstable ($O(N_p^2)$ joint perturbations) while unary signals enjoy greater stability ($O(N_p)$ perturbations, CLT). From three design principles derived from this diagnosis, we construct CATIS as a constructive validation: unary signals raise the trigger threshold, triage suppresses the gain. On ViT-Large at 63% FLOPs reduction, CATIS retains 96.9% of vanilla accuracy (81.0%) on ImageNet-1K where all baselines collapse to 43--65%.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑