精炼与对齐：视觉问答中通过多智能体交互实现置信度校准
Refine and Align: Confidence Calibration through Multi-Agent Interaction in VQA
AI总结：
针对VQA系统易输出过度自信回答、置信度估计可靠性不足的问题，本文提出多智能体辩论框架AlignVQA及可微校准感知损失aligncal，可有效提升置信度与预测实际正确性的对齐程度，降低校准误差。
摘要：
在视觉问答（VQA）与智能体AI场景下，校准指的是AI系统对其答案的置信度与答案实际正确程度的匹配水平。当这类系统自主运行、必须在视觉不确定性条件下决策时，校准特性尤为重要。尽管依托先进视觉语言模型（VLM）的现代VQA系统因精度提升，被越来越多地应用于医疗诊断、自主导航等高风险领域，但其置信度估计的可靠性仍未得到充分研究，尤其是这类系统往往会生成过度自信的回答。为解决该问题，我们提出基于辩论的多智能体框架AlignVQA：采用不同提示策略的多样化专用VLM生成候选答案后，进入两阶段交互流程，由通才智能体对候选方案进行批判、精炼与聚合，该辩论过程可生成更准确反映模型真实预测性能的置信度估计。我们发现，校准效果更好的专用智能体能产出对齐度更高的置信度。此外，我们提出名为aligncal的新型可微校准感知损失函数，通过最小化校准误差的上界来微调专用智能体，该目标可显式提升每个智能体置信度估计的保真度。在多个VQA基准数据集上的实验结果证实了本方法的有效性，表明其可大幅降低校准偏差。
英文摘要：
In the context of Visual Question Answering (VQA) and Agentic AI, calibration refers to how closely an AI system's confidence in its answers reflects their actual correctness. This aspect becomes especially important when such systems operate autonomously and must make decisions under visual uncertainty. While modern VQA systems, powered by advanced vision-language models (VLMs), are increasingly used in high-stakes domains like medical diagnostics and autonomous navigation due to their improved accuracy, the reliability of their confidence estimates remains under-examined. Particularly, these systems often produce overconfident responses. To address this, we introduce AlignVQA, a debate-based multi-agent framework, in which diverse specialized VLM -- each following distinct prompting strategies -- generate candidate answers and then engage in two-stage interaction: generalist agents critique, refine and aggregate these proposals. This debate process yields confidence estimates that more accurately reflect the model's true predictive performance. We find that more calibrated specialized agents produce better aligned confidences. Furthermore, we introduce a novel differentiable calibration-aware loss function called aligncal designed to fine-tune the specialized agents by minimizing an upper bound on the calibration error. This objective explicitly improves the fidelity of each agent's confidence estimates. Empirical results across multiple benchmark VQA datasets substantiate the efficacy of our approach, demonstrating substantial reductions in calibration discrepancies. Furthermore, we propose a novel differentiable calibration-aware loss to fine-tune the specialized agents and improve the quality of their individual confidence estimates based on minimising upper bound calibration error.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
