CAMD：面向多模态大语言模型高效推理的覆盖感知多模态解码
CAMD: Coverage-Aware Multimodal Decoding for Efficient Reasoning of Multimodal Large Language Models
AI总结：
本文提出CAMD，通过动态分配计算资源提升多模态大语言模型的推理效率与可靠性，解决解码方法在易例和难例间的计算不匹配问题。
摘要：
近年来，多模态大语言模型（MLLMs）在视觉-语言任务中展示了出色的推理能力，但仍面临计算难度不匹配的挑战。通过实证分析，我们发现现有解码方法可能在易例上浪费计算资源而在难例上不足，影响模型效果和效率。为此，我们首先开发了一个理论框架，将采样覆盖、实例难度和残余风险联系起来。分析表明，多模态推理表现出重尾难度分布；一小部分难或模糊样本主导残余失败概率。基于此洞察，我们提出覆盖感知多模态解码（CAMD），一种自适应推理机制，根据估计的不确定性动态分配计算资源。CAMD集成了证据加权评分、后验覆盖估计和顺序贝叶斯更新，以在有限token预算下平衡效率和可靠性。在各种基准数据集和基线上的实验验证了本方法的有效性和优势。
英文摘要：
Recent advances in Multimodal Large Language Models (MLLMs) have shown impressive reasoning capabilities across vision-language tasks, yet still face the challenge of compute-difficulty mismatch. Through empirical analyses, we identify that existing decoding methods may waste compute on easy cases while underserving hard ones, affecting both model effectiveness and efficiency. To address this issue, we first develop a theoretical framework that links sampling coverage, instance difficulty, and residual risk. Our analysis reveals that multimodal reasoning exhibits a heavy-tailed difficulty distribution; a small subset of hard or ambiguous samples dominates the residual failure probability. Based on this insight, we propose Coverage-Aware Multimodal Decoding (CAMD), an adaptive inference mechanism that dynamically allocates computation according to estimated uncertainty. CAMD integrates evidence-weighted scoring, posterior coverage estimation, and sequential Bayesian updating to balance efficiency and reliability under a limited token budget. Experiments on various benchmark datasets and baselines demonstrate the effectiveness and advantages of our approach.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
