基于检测的中间监督用于视觉问答
Detection-based Intermediate Supervision for Visual Question Answering
AI总结：
提出基于检测的中间监督（DIS）方法，通过生成式检测框架改进神经模块网络的中间监督，提升组合视觉问答的准确性和推理一致性。
摘要：
近年来，神经模块网络（NMNs）在回答组合性视觉问题方面取得了持续成功，尤其是那些涉及多跳视觉和逻辑推理的问题。NMNs 利用问题推理路径中的实例模块将复杂问题分解为多个子任务，然后利用中间监督来指导答案预测，从而提高推理的可解释性。然而，由于对中间监督的建模较为粗略，其性能可能受到阻碍。例如，（1）一个先验假设是每个实例模块仅指代一个接地对象，却忽略了其他可能相关的接地对象，这阻碍了充分的跨模态对齐学习；（2）基于 IoU 的中间监督可能引入噪声信号，因为边界框重叠问题可能将模型的注意力引导至无关对象。为解决这些问题，提出了一种新方法，即基于检测的中间监督（DIS），它采用生成式检测框架，通过序列生成促进多个接地监督。因此，DIS 提供了更全面和准确的中间监督，从而提升了答案预测性能。此外，通过考虑中间结果，DIS 增强了回答组合性问题及其子问题的一致性。大量实验证明了所提出的 DIS 的优越性，与先前方法相比，展示了更高的准确性和最先进的推理一致性。
英文摘要：
Recently, neural module networks (NMNs) have yielded ongoing success in answering compositional visual questions, especially those involving multi-hop visual and logical reasoning. NMNs decompose the complex question into several sub-tasks using instance-modules from the reasoning paths of that question and then exploit intermediate supervisions to guide answer prediction, thereby improving inference interpretability. However, their performance may be hindered due to sketchy modeling of intermediate supervisions. For instance, (1) a prior assumption that each instance-module refers to only one grounded object yet overlooks other potentially associated grounded objects, impeding full cross-modal alignment learning; (2) IoU-based intermediate supervisions may introduce noise signals as the bounding box overlap issue might guide the model's focus towards irrelevant objects. To address these issues, a novel method, \textbf{\underline{D}}etection-based \textbf{\underline{I}}ntermediate \textbf{\underline{S}}upervision (DIS), is proposed, which adopts a generative detection framework to facilitate multiple grounding supervisions via sequence generation. As such, DIS offers more comprehensive and accurate intermediate supervisions, thereby boosting answer prediction performance. Furthermore, by considering intermediate results, DIS enhances the consistency in answering compositional questions and their sub-questions.Extensive experiments demonstrate the superiority of our proposed DIS, showcasing both improved accuracy and state-of-the-art reasoning consistency compared to prior approaches.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
