当选择成为先验：用于科学图表多选问答的对比解码
BUZZY: Contrastive Scoring to Mitigate Text-Induced Bias in Multimodal Multiple-Choice QA
AI总结：
本文提出SCICON方法，通过对比文本和图像信息，减少选择偏差，提升科学图表多选问答的准确性。
摘要：
科学图表多选问答需要模型在多样化的视觉证据上进行推理，从图表和多面板图到显微镜和生物医学图像。然而，这种设置存在独特的偏差：答案选项本身可以作为先验，引导多模态模型倾向于科学合理的选项，即使图表支持不同的答案。我们通过一个简单的问题进行探讨：如果解码明确地减少模型单独从文本中偏好的内容，从而更倾向于基于图像的证据？为此，我们提出了SCICON，一种无需训练的解码方法，通过将候选选项的得分减去纯文本选项的得分来评分。与以往通过对比原始输入与扭曲图像或扰动指令来缓解幻觉的对比解码方法不同，SCICON直接针对候选文本中编码的选择诱导先验。在三个科学图表问答基准和三个模型主干上，SCICON在标准解码基线上始终提高了准确性。这些结果表明，对抗选择诱导的先验的解码是一种有效且简单的提升科学多选问答中基于图像推理的方法。
英文摘要：
Multimodal multiple-choice question answering (MCQA) provides a standardized and objectively measurable setting for evaluating vision-language models (VLMs). However, because the MCQA format incorporates the candidate choices into the input context, it introduces several unintended biases. Previous work has primarily focused on structural biases, such as preferences for certain choices. Instead, we argue that the choices act as textual priors, causing models to favor linguistically plausible options regardless of the visual content. We hypothesize and empirically verify that a model genuinely relies on visual evidence only when its multimodal distribution significantly diverges from its text-only distribution. Based on this observation we propose BUZZY,a training-free decoding method that corrects multimodal predictions by subtracting the text-only distribution. Experiments with five VLMs on five multimodal MCQA benchmarks demonstrate that BUZZY achieves the highest average accuracy among state-of-the-art methods while reducing inference latency by over 28% compared to prior contrastive decoding approaches. Overall, these results suggest that amplifying the visual signal by penalizing text-only preferences is key to efficient and robust multimodal MCQA reasoning. Code and additional resources are provided https://txxnrd.github.io/buzzy/.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
