arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2602.01345cs.CV
收藏

基于双链接熵分析的自适应视觉自回归加速

Adaptive Visual Autoregressive Acceleration via Dual-Linkage Entropy Analysis

Tongji University(同济大学);Harbin Institute of Technology(哈尔滨工业大学);Shandong Academy of Sciences(山东省科学院);Macquarie University(麦考瑞大学)

Yu Zhang, Jingyi Liu, Feng Liu, Duoqian Miao, Qi Zhang, Kexue Fu, Changwei Wang, Longbing Cao

AI总结:

NOVA通过熵分析提出无需训练的VAR模型token减少加速框架,通过动态调整尺度和层的token减少比率,实现推理加速与生成质量的平衡。

摘要:

视觉自回归建模(VAR)由于涉及大量的token数量而面临显著的计算成本。未能考虑建模动态的持续演变,现有VAR token减少方法面临三个关键限制:启发式阶段划分、非适应性调度和有限的加速范围,从而留下了巨大的加速潜力。由于熵变本质上反映了预测不确定性的转变,它提供了一种原则性的度量来捕捉建模动态演变。因此,我们提出NOVA,通过熵分析提出一种无需训练的token减少加速框架,用于VAR模型。NOVA在推理过程中通过在线识别尺度熵增长的拐点来适应性地确定加速激活规模。通过尺度链接和层链接比率调整,NOVA动态计算每个尺度和层的不同token减少比率,修剪低熵token,同时利用先前尺度残差导出的cache来加速推理并维持生成质量。广泛的实验和分析验证了NOVA作为一种简单而有效的无需训练的加速框架。

英文摘要:

Visual AutoRegressive modeling (VAR) suffers from substantial computational cost due to the massive token count involved. Failing to account for the continuous evolution of modeling dynamics, existing VAR token reduction methods face three key limitations: heuristic stage partition, non-adaptive schedules, and limited acceleration scope, thereby leaving significant acceleration potential untapped. Since entropy variation intrinsically reflects the transition of predictive uncertainty, it offers a principled measure to capture modeling dynamics evolution. Therefore, we propose NOVA, a training-free token reduction acceleration framework for VAR models via entropy analysis. NOVA adaptively determines the acceleration activation scale during inference by online identifying the inflection point of scale entropy growth. Through scale-linkage and layer-linkage ratio adjustment, NOVA dynamically computes distinct token reduction ratios for each scale and layer, pruning low-entropy tokens while reusing the cache derived from the residuals at the prior scale to accelerate inference and maintain generation quality. Extensive experiments and analyses validate NOVA as a simple yet effective training-free acceleration framework.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑