arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2205.02671cs.CVcs.AIcs.LG
收藏

对我来说正确的东西还不适合你:通过多任务学习确定相对方向的数据集

What is Right for Me is Not Yet Right for You: A Dataset for Grounding Relative Directions via Multi-Task Learning

University of Hamburg(汉堡大学)

Jae Hee Lee, Matthias Kerzel, Kyra Ahrens, Cornelius Weber, Stefan Wermter

摘要:

理解空间关系对于智能体在物理世界中行动和通信至关重要。相对方向是描述目标对象相对于参考对象的内在取向的相对位置的空间关系。接地相对方向比接地绝对方向更困难,因为它不仅需要一个模型来检测图像中的对象并基于此信息识别空间关系,而且还需要识别对象的方向并将此信息集成到推理过程中。我们研究了具有挑战性的问题接地相对方向与端到端神经网络。为此,我们提供了GRiD-3D,这是一种新的数据集,具有相对方向,并补充了现有的视觉问答(VQA)数据集,如CLEVR,只涉及绝对方向。我们还为数据集提供了两个已建立的端到端VQA模型的基线。实验评估表明,当数据集中的问题模拟接地相对方向的必要子任务时,回答相对方向的问题是可行的。我们发现,这些子任务的学习顺序反映了处理相对方向的直观管道的步骤。

英文摘要:

Understanding spatial relations is essential for intelligent agents to act and communicate in the physical world. Relative directions are spatial relations that describe the relative positions of target objects with regard to the intrinsic orientation of reference objects. Grounding relative directions is more difficult than grounding absolute directions because it not only requires a model to detect objects in the image and to identify spatial relation based on this information, but it also needs to recognize the orientation of objects and integrate this information into the reasoning process. We investigate the challenging problem of grounding relative directions with end-to-end neural networks. To this end, we provide GRiD-3D, a novel dataset that features relative directions and complements existing visual question answering (VQA) datasets, such as CLEVR, that involve only absolute directions. We also provide baselines for the dataset with two established end-to-end VQA models. Experimental evaluations show that answering questions on relative directions is feasible when questions in the dataset simulate the necessary subtasks for grounding relative directions. We discover that those subtasks are learned in an order that reflects the steps of an intuitive pipeline for processing relative directions.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑