UNMuTe：统一导航与多模态对话式文本生成
UNMuTe: Unifying Navigation and Multimodal Dialogue-like Text Generation
AI总结：
UNMuTe提出一种结合GPT-2对话模型与视觉语言导航器的智能体，通过主动提问获取路径指引，在CVDN和NDH任务上取得最先进性能。
摘要：
智能自主智能体在机器人、自动驾驶车辆等各类现实应用中正变得越来越重要。这些智能体必须具备的一项关键技能是与其周围实体（如其他智能体或人类）进行交互的能力。在本工作中，我们旨在构建一个能够在环境中高效导航，同时能够与一个先知（或人类）以自然语言进行交互，并在对自身导航表现不确定时询问方向的智能体。交互由智能体发起，它生成一个问题，随后先知基于到达目标的最短轨迹来回答该问题。该过程可以在导航过程中多次执行，从而使智能体能够与先知保持对话。为此，我们提出了一种名为UNMuTe的新型计算模型，该模型由两个主要组件组成：一个对话模型和一个导航器。具体而言，对话模型基于GPT-2解码器，用于处理由文本和图像组成的多模态数据。首先，对话模型被训练用于生成问答对：问题利用当前图像生成，而答案则利用通往目标路径上的未来图像生成。随后，训练一个VLN模型来跟随对话，预测导航动作，或在需要帮助时触发对话模型。在我们的实验分析中，我们表明UNMuTe在涉及对话的主要导航任务上取得了最先进的性能，即协作视觉与对话导航（CVDN）和基于对话历史的导航（NDH），证明我们的方法能够有效生成有用的问题和答案来引导导航。
英文摘要：
Smart autonomous agents are becoming increasingly important in various real-life applications, including robotics and autonomous vehicles. One crucial skill that these agents must possess is the ability to interact with their surrounding entities, such as other agents or humans. In this work, we aim at building an intelligent agent that can efficiently navigate in an environment while being able to interact with an oracle (or human) in natural language and ask for directions when it is unsure about its navigation performance. The interaction is started by the agent that produces a question, which is then answered by the oracle on the basis of the shortest trajectory to the goal. The process can be performed multiple times during navigation, thus enabling the agent to hold a dialogue with the oracle. To this end, we propose a novel computational model, named UNMuTe, that consists of two main components: a dialogue model and a navigator. Specifically, the dialogue model is based on a GPT-2 decoder that handles multimodal data consisting of both text and images. First, the dialogue model is trained to generate question-answer pairs: the question is generated using the current image, while the answer is produced leveraging future images on the path toward the goal. Subsequently, a VLN model is trained to follow the dialogue predicting navigation actions or triggering the dialogue model if it needs help. In our experimental analysis, we show that UNMuTe achieves state-of-the-art performance on the main navigation tasks implying dialogue, i.e. Cooperative Vision and Dialogue Navigation (CVDN) and Navigation from Dialogue History (NDH), proving that our approach is effective in generating useful questions and answers to guide navigation.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
