在何时不回答：评估多模态推理系统中的退避
Knowing When Not to Answer: Evaluating Abstention in Multimodal Reasoning Systems
AI总结：
本文提出MM-AQA基准，通过视觉模态依赖性和证据充分性变换生成不可回答实例，评估三种前沿VLM和两种MAS架构，发现VLM在标准提示下 rarely 退避，MAS提升退避但引入准确率-退避权衡，序列设计表现优异，表明退避意识训练比提示或更多代理更重要。
摘要：
有效的退避（EA），识别证据不足并拒绝回答，对可靠多模态系统至关重要。然而现有评估范式假设可回答性，迫使模型始终回应。退避已在文本-only场景研究但多模态仍不充分；当前基准要么忽略不可回答性或依赖粗略方法错过真实失败模式。我们引入MM-AQA基准，通过两个轴向变换从可回答实例生成不可回答实例。评估三个前沿VLM（涵盖闭源和开源模型）和两种MAS架构在2079个样本上，发现：（1）标准提示下VLM rarely 退避；即使简单置信度基线也优于此设置；（2）MAS提升退避但引入准确率-退避权衡；（3）序列设计匹配或超过迭代变体，表明瓶颈是退避意识而非推理深度；（4）模型在图像或文本证据缺失时退避，但尝试与退化或矛盾证据调和。有效的多模态退避需要退避意识训练而非更好的提示或更多代理。
英文摘要：
Effective abstention (EA), recognizing evidence insufficiency and refraining from answering, is critical for reliable multimodal systems. Yet existing evaluation paradigms for vision-language models (VLMs) and multi-agent systems (MAS) assume answerability, pushing models to always respond. Abstention has been studied in text-only settings but remains underexplored multimodally; current benchmarks either ignore unanswerability or rely on coarse methods that miss realistic failure modes. We introduce MM-AQA, a benchmark that constructs unanswerable instances from answerable ones via transformations along two axes: visual modality dependency and evidence sufficiency. Evaluating three frontier VLMs spanning closed and open-source models and two MAS architectures across 2079 samples, we find: (1) under standard prompting, VLMs rarely abstain; even simple confidence baselines outperform this setup, (2) MAS improves abstention but introduces an accuracy-abstention trade-off, (3) sequential designs match or exceed iterative variants, suggesting the bottleneck is miscalibration rather than reasoning depth, and (4) models abstain when image or text evidence is absent, but attempt reconciliation with degraded or contradictory evidence. Effective multimodal abstention requires abstention-aware training rather than better prompting or more agents.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
