语言与草图：一个由 LLM 驱动的交互式多模态多任务机器人导航框架
Language and Sketching: An LLM-driven Interactive Multimodal Multitask Robot Navigation Framework
AI总结：
提出由LLM驱动的交互式多模态多任务机器人导航框架LIM2N，通过语言和手绘输入作为导航约束，结合强化学习智能体处理多任务，实现多模态输入推理、多任务规划与智能感知模块的协同，提升了用户需求理解与交互体验。
摘要：
社交感知导航系统已发展到能够在执行多种任务（如点对点导航、跟随人类和引导人类）的同时熟练避开各种障碍物。然而，一个显著的差距依然存在：在人机交互（HRI）中，向机器人传达指令的过程需要复杂的数学公式。此外，任务之间的转换并不具备人们所期望的直观控制和以用户为中心的交互性。在这项工作中，我们提出了一个由 LLM 驱动的交互式多模态多任务机器人导航框架，称为 LIM2N，以解决上述导航领域的新挑战。我们首先引入一个多模态交互框架，其中语言和手绘输入可以作为导航约束和控制目标。接着，构建一个强化学习智能体，利用接收到的信息处理多个任务。关键在于，LIM2N 在复杂系统中实现了多模态输入推理、多任务规划以及智能感知模块的适应与处理之间的顺畅协作。在仿真和真实世界中进行了大量实验，证明 LIM2N 具备卓越的用户需求理解能力，并带来增强的交互体验。
英文摘要：
The socially-aware navigation system has evolved to adeptly avoid various obstacles while performing multiple tasks, such as point-to-point navigation, human-following, and -guiding. However, a prominent gap persists: in Human-Robot Interaction (HRI), the procedure of communicating commands to robots demands intricate mathematical formulations. Furthermore, the transition between tasks does not quite possess the intuitive control and user-centric interactivity that one would desire. In this work, we propose an LLM-driven interactive multimodal multitask robot navigation framework, termed LIM2N, to solve the above new challenge in the navigation field. We achieve this by first introducing a multimodal interaction framework where language and hand-drawn inputs can serve as navigation constraints and control objectives. Next, a reinforcement learning agent is built to handle multiple tasks with the received information. Crucially, LIM2N creates smooth cooperation among the reasoning of multimodal input, multitask planning, and adaptation and processing of the intelligent sensing modules in the complicated system. Extensive experiments are conducted in both simulation and the real world demonstrating that LIM2N has superior user needs understanding, alongside an enhanced interactive experience.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
