arXiv 2601.138792026-03-11cs.MMcs.CLcs.CV

链式推理压缩不应盲目：通过双路径锚定实现高效的多模态推理的V-Skip

Chain-of-Thought Compression Should Not Be Blind: V-Skip for Efficient Multimodal Reasoning via Dual-Path Anchoring

School of Software Engineering, Xi’an Jiaotong University（西安交通大学软件工程学院）；Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；Shanghai Artificial Intelligence Laboratory（上海人工智能实验室）；Harbin Institute of Technology, Shenzhen（深圳哈尔滨工业大学）；University of Science and Technology Beijing（北京科技大学）

Dongxu Zhang, Yiding Sun, Cheng Tan, Wenbiao Yan, Ning Yang, Jihua Zhu, Haijun Zhang

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

V-Skip通过双路径锚定机制解决多模态推理中令牌剪枝的盲目性问题，实现高效的推理速度提升与精度保持。

摘要：

尽管链式推理（CoT）推理显著提升了多模态大语言模型（MLLMs）的性能，但其自回归性质导致了可接受的延迟限制。目前试图通过令牌压缩来缓解这一问题的努力往往失败，因为盲目地将文本中心的指标应用于多模态上下文。我们识别出一种关键的失败模式，称为视觉遗忘，其中语言上冗余的令牌被错误地剪枝，导致幻觉。为了解决这个问题，我们引入了V-Skip，将令牌剪枝重新表述为视觉锚定信息瓶颈（VA-IB）优化问题。V-Skip采用双路径门控机制，通过语言意外和跨模态注意力流来加权令牌的重要性，有效地拯救了视觉显著的锚点。在Qwen2-VL和Llama-3.2家族上的大量实验表明，V-Skip实现了2.9倍的速度提升，同时精度损失 negligible。具体来说，它保留了细粒度的视觉细节，在DocVQA上优于其他基线30%以上。

英文摘要：

While Chain-of-Thought (CoT) reasoning significantly enhances the performance of Multimodal Large Language Models (MLLMs), its autoregressive nature incurs prohibitive latency constraints. Current efforts to mitigate this via token compression often fail by blindly applying text-centric metrics to multimodal contexts. We identify a critical failure mode termed Visual Amnesia, where linguistically redundant tokens are erroneously pruned, leading to hallucinations. To address this, we introduce V-Skip that reformulates token pruning as a Visual-Anchored Information Bottleneck (VA-IB) optimization problem. V-Skip employs a dual-path gating mechanism that weighs token importance through both linguistic surprisal and cross-modal attention flow, effectively rescuing visually salient anchors. Extensive experiments on Qwen2-VL and Llama-3.2 families demonstrate that V-Skip achieves a $2.9\times$ speedup with negligible accuracy loss. Specifically, it preserves fine-grained visual details, outperforming other baselines over 30\% on the DocVQA.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。