arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 1806.02453cs.CV
收藏

基于渐进模块网络的视觉推理

Visual Reasoning by Progressive Module Networks

University of Toronto(多伦多大学);Vector Institute(向量研究所);NVIDIA(英伟达)

Seung Wook Kim, Makarand Tapaswi, Sanja Fidler

摘要:

人类通过建立在先前获得的知识之上来学习解决日益复杂的任务。通常情况下,我们学习的任务中存在一个自然的进展-大多数不需要完全独立的解决方案,但可以分解为更简单的子任务。我们建议将每个任务的求解器表示为神经模块,该神经模块以类似函数式程序的方式调用现有模块(简单任务的求解器)。下层模块对于调用模块来说是一个黑盒,只能通过查询和输出进行通信。因此,一个新任务的模块学习查询现有的模块,并组合它们的输出,以产生自己的输出。我们的模型有效地结合了以前的技能,不会忘记,并且是完全可区分的。我们测试了我们的模型在学习一组视觉推理任务,并证明在所有任务的学习逐步提高性能。通过使用人类法官评估推理过程,我们表明,我们的模型比基于注意力的基线更具可解释性。

英文摘要:

Humans learn to solve tasks of increasing complexity by building on top of previously acquired knowledge. Typically, there exists a natural progression in the tasks that we learn - most do not require completely independent solutions, but can be broken down into simpler subtasks. We propose to represent a solver for each task as a neural module that calls existing modules (solvers for simpler tasks) in a functional program-like manner. Lower modules are a black box to the calling module, and communicate only via a query and an output. Thus, a module for a new task learns to query existing modules and composes their outputs in order to produce its own output. Our model effectively combines previous skill-sets, does not suffer from forgetting, and is fully differentiable. We test our model in learning a set of visual reasoning tasks, and demonstrate improved performances in all tasks by learning progressively. By evaluating the reasoning process using human judges, we show that our model is more interpretable than an attention-based baseline.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑