arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2203.06822cs.CVcs.CLcs.RO
收藏

基于特定区域动态层注意力的自主车辆接地命令

Grounding Commands for Autonomous Vehicles via Layer Fusion with Region-specific Dynamic Layer Attention

University of Macau(澳门大学)

Hou Pong Chan, Mingxi Guo, Cheng-Zhong Xu

摘要:

将命令接地到视觉环境是自动驾驶车辆与人类之间交互的重要组成部分。在这项工作中,我们研究了自动驾驶汽车的语言接地问题,其目的是根据乘客的自然语言命令在视觉场景中定位一个区域。以前的工作只采用视觉和语言预训练模型的顶层表示来预测命令所指的区域。然而,这样的方法省略了在其他层中编码的有用特征,并且因此导致对输入场景和命令的不充分理解。为了解决这一限制,我们提出了该任务的第一层融合方法。由于不同的视觉区域可能需要不同类型的功能,以消除彼此的歧义,我们进一步提出了区域特定的动态(RSD)层的注意力,以适应性地融合跨层的多模态信息为每个区域。在Talk2Car基准上进行的大量实验表明,我们的方法有助于预测更准确的区域,并且优于最先进的方法。

英文摘要:

Grounding a command to the visual environment is an essential ingredient for interactions between autonomous vehicles and humans. In this work, we study the problem of language grounding for autonomous vehicles, which aims to localize a region in a visual scene according to a natural language command from a passenger. Prior work only employs the top layer representations of a vision-and-language pre-trained model to predict the region referred to by the command. However, such a method omits the useful features encoded in other layers, and thus results in inadequate understanding of the input scene and command. To tackle this limitation, we present the first layer fusion approach for this task. Since different visual regions may require distinct types of features to disambiguate them from each other, we further propose the region-specific dynamic (RSD) layer attention to adaptively fuse the multimodal information across layers for each region. Extensive experiments on the Talk2Car benchmark demonstrate that our approach helps predict more accurate regions and outperforms state-of-the-art methods.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑