arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2009.10054cs.CVcs.LG
收藏

视觉问答中注意网络的正则化及其异常检测

Regularizing Attention Networks for Anomaly Detection in Visual Question Answering

POSTECH(浦项科技大学);Kakao Brain

Doyup Lee, Yeongjae Cheon, Wook-Shin Han

摘要:

为了真实世界应用的稳定性和可靠性,DNN在单峰任务中的鲁棒性已经被评估。然而,很少有研究认为异常情况下,一个视觉问答(VQA)模型可能会遇到在测试时部署在现实世界中。在这项研究中,我们评估了最先进的VQA模型对五种不同异常的鲁棒性,包括最坏情况下,最常见的情况下,VQA模型的当前限制。与单峰任务的结果不同,VQA模型中答案的最大置信度不能检测异常输入,并且输出的后训练,如离群值暴露,对VQA模型无效。因此,我们提出了一种基于注意力的方法,它使用输入图像和问题之间的推理的信心,并显示出更有前途的结果比以前的方法在单峰任务。此外,我们表明,注意力网络的最大熵正则化可以显着提高基于注意力的异常检测的VQA模型。由于简单性,基于注意力的异常检测和正则化是模型不可知的方法,可以用于最先进的VQA模型中的各种跨模态注意力。结果表明,跨模态注意在VQA是重要的,不仅提高VQA的准确性,但也对各种异常的鲁棒性。

英文摘要:

For stability and reliability of real-world applications, the robustness of DNNs in unimodal tasks has been evaluated. However, few studies consider abnormal situations that a visual question answering (VQA) model might encounter at test time after deployment in the real-world. In this study, we evaluate the robustness of state-of-the-art VQA models to five different anomalies, including worst-case scenarios, the most frequent scenarios, and the current limitation of VQA models. Different from the results in unimodal tasks, the maximum confidence of answers in VQA models cannot detect anomalous inputs, and post-training of the outputs, such as outlier exposure, is ineffective for VQA models. Thus, we propose an attention-based method, which uses confidence of reasoning between input images and questions and shows much more promising results than the previous methods in unimodal tasks. In addition, we show that a maximum entropy regularization of attention networks can significantly improve the attention-based anomaly detection of the VQA models. Thanks to the simplicity, attention-based anomaly detection and the regularization are model-agnostic methods, which can be used for various cross-modal attentions in the state-of-the-art VQA models. The results imply that cross-modal attention in VQA is important to improve not only VQA accuracy, but also the robustness to various anomalies.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑