arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2412.08263cs.CL
收藏

面向可解释基于图的视觉问答的离散子图采样

Discrete Subgraph Sampling for Interpretable Graph based Visual Question Answering

University of Stuttgart(斯图加特大学)

Pascal Tilli, Ngoc Thang Vu

AI总结:

针对可解释视觉问答中内在解释方法稀缺的问题,研究将离散子集采样整合进图基视觉问答系统,在GQA数据集上验证其缓解性能权衡的效果,并通过人工评估证实词元共现指标与人类偏好强相关。

摘要:

可解释人工智能(XAI)旨在提升机器学习模型的透明度。现有诸多方法聚焦于事后生成解释,而在生成预测的同时内在生成解释的可解释方法相对较少。本研究将多种离散子集采样方法整合进基于图的视觉问答系统,对比它们在内在生成可解释性子图方面的效果。我们在GQA数据集上评估了这些方法,结果表明整合的方法有效缓解了可解释性与答案准确率之间的性能权衡,同时实现了答案词元与问题词元的强共现。此外,我们采用扩展Bradley-Terry模型的对比设置开展人工评估,以衡量生成子图的可解释性,结果显示答案与问题词元共现指标与人类偏好高度相关。我们的源代码已公开。

英文摘要:

Explainable artificial intelligence (XAI) aims to make machine learning models more transparent. While many approaches focus on generating explanations post-hoc, interpretable approaches, which generate the explanations intrinsically alongside the predictions, are relatively rare. In this work, we integrate different discrete subset sampling methods into a graph-based visual question answering system to compare their effectiveness in generating interpretable explanatory subgraphs intrinsically. We evaluate the methods on the GQA dataset and show that the integrated methods effectively mitigate the performance trade-off between interpretability and answer accuracy, while also achieving strong co-occurrences between answer and question tokens. Furthermore, we conduct a human evaluation to assess the interpretability of the generated subgraphs using a comparative setting with the extended Bradley-Terry model, showing that the answer and question token co-occurrence metrics strongly correlate with human preferences. Our source code is publicly available.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑