arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 1811.08481cs.CV
收藏

VQA无问答培训

VQA with no questions-answers training

Weizmann Institute of Science(魏茨曼科学研究所)

Ben-Zion Vatashsky, Shimon Ullman

摘要:

近年来,教机器回答视觉问题的方法取得了重大进展,但目前的方法仍然缺乏重要的人类能力,包括以模块化方式集成新的视觉类和概念,为答案提供解释,以及在没有明确示例的情况下处理新领域。我们提出了一种新的方法,包括两个主要部分:生成一个问题图表示,和回答过程,引导的抽象结构的问题图调用一组可扩展的视觉估计。训练是针对语言部分和视觉部分本身进行的,但与现有方案不同,该方法不需要使用具有相关问题和答案的图像进行任何训练。这种方法是能够处理新的域(扩展的问题类型和新的对象类,属性和关系),只要相应的视觉估计。此外,它还可以为答案提供解释,并在问题不以图像为基础时提出替代方案。我们证明,这种方法实现了高性能和域的可扩展性,没有任何问答训练。

英文摘要:

Methods for teaching machines to answer visual questions have made significant progress in recent years, but current methods still lack important human capabilities, including integrating new visual classes and concepts in a modular manner, providing explanations for the answers and handling new domains without explicit examples. We propose a novel method that consists of two main parts: generating a question graph representation, and an answering procedure, guided by the abstract structure of the question graph to invoke an extendable set of visual estimators. Training is performed for the language part and the visual part on their own, but unlike existing schemes, the method does not require any training using images with associated questions and answers. This approach is able to handle novel domains (extended question types and new object classes, properties and relations) as long as corresponding visual estimators are available. In addition, it can provide explanations to its answers and suggest alternatives when questions are not grounded in the image. We demonstrate that this approach achieves both high performance and domain extensibility without any questions-answers training.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑