arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2602.04924cs.LGcs.SD
收藏

何时回答:可靠音频-视觉问答的自适应置信度细化

Knowing When to Answer: Adaptive Confidence Refinement for Reliable Audio-Visual Question Answering

School of Computing, KAIST, Republic of Korea(韩国釜山科学技术院计算机科学学院);Laval University(拉瓦尔大学);Mila-Quebec AI Institute(魁北克人工智能研究所)

Dinh Phu Tran, Jihoon Jeong, Saad Wazir, Seongah Kim, Thao Do, Cem Subakan, Daeyoung Kim

AI总结:

本文提出自适应置信度细化方法,用于提升音频-视觉问答任务中可靠性的性能。

摘要:

我们提出了一个正式的问题表述,用于可靠的音频-视觉问答(R-AVQA),其中我们更倾向于回避而非错误回答。尽管最近的AVQA模型具有较高的准确性,但它们识别自己可能错误时的能力以及因此回避回答的能力仍然是研究的空白领域。为了填补这一空白,我们探索了几种方法,然后提出了自适应置信度细化(ACR),一种轻量级的方法,用于进一步提高R-AVQA的性能。我们的关键见解是最大softmax概率(MSP)在强校准下是贝叶斯最优的,但这种情况通常在深度神经网络中不成立,特别是在多模态模型中。我们没有取代MSP,而是将其保持为主要的置信度信号,并在MSP被认为不可靠时应用输入自适应的残差修正。ACR引入了两个学习头:i)一个残差风险头,预测MSP未捕捉到的低幅度正确性残差,以及ii)一个置信度门控头,以确定MSP的可信度。我们的实验和理论分析表明,ACR在三种不同的AVQA架构中,在分布内和分布外以及数据偏置设置上均优于现有方法,为R-AVQA任务建立了坚实的基础。代码和检查点将在接受后提供,详见https://github.com/PhuTran1005/R-AVQA。

英文摘要:

We present a formal problem formulation for \textit{Reliable} Audio-Visual Question Answering ($\mathcal{R}$-AVQA), where we prefer abstention over answering incorrectly. While recent AVQA models have high accuracy, their ability to identify when they are likely wrong and their consequent abstention from answering remain underexplored areas of research. To fill this gap, we explore several approaches and then propose Adaptive Confidence Refinement (ACR), a lightweight method to further enhance the performance of $\mathcal{R}$-AVQA. Our key insight is that the Maximum Softmax Probability (MSP) is Bayes-optimal only under strong calibration, a condition usually not met in deep neural networks, particularly in multimodal models. Instead of replacing MSP, our ACR maintains it as a primary confidence signal and applies input-adaptive residual corrections when MSP is deemed unreliable. ACR introduces two learned heads: i) a Residual Risk Head that predicts low-magnitude correctness residuals that MSP does not capture, and ii) a Confidence Gating Head to determine MSP trustworthiness. Our experiments and theoretical analysis show that ACR consistently outperforms existing methods on in- and out-of-disrtibution, and data bias settings across three different AVQA architectures, establishing a solid foundation for $\mathcal{R}$-AVQA task. The code and checkpoints will be available upon acceptance \href{https://github.com/PhuTran1005/R-AVQA}{at here}

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑