Prompter：利用大语言模型提示实现数据高效的具身指令跟随
Prompter: Utilizing Large Language Model Prompting for a Data Efficient Embodied Instruction Following
AI总结：
针对具身指令跟随任务，提出Prompter模块化方法，嵌入机器人物理约束并以大语言模型提示替代专用训练模型实现物体搜索，在ALFRED基准上性能显著超越现有最优水平。
摘要：
具身指令跟随（Embodied Instruction Following, EIF）研究如何控制自主移动操作机器人完成自然语言指令描述的长时序任务。尽管大量EIF研究在模拟器中开展，但该领域的最终目标是将智能体部署到现实场景中，这也是近期方法摒弃端到端模型训练、转而采用无需昂贵专家操作数据的模块化方法的原因之一。不过，将模块化思路引入EIF仍处于早期阶段，探寻适用于EIF任务的有效模块尚未形成定论。\n本文提出利用两类外部来源的知识拓展模块化设计。首先，我们证明将部署机器人的物理约束嵌入模块设计效果显著，该设计还能让同一模块化系统仅需极少修改，即可在不同配置的机器人上运行。其次，我们表明以往由需要专用数据集的训练模型实现的基于地标的物体搜索，可替换为通过提示预训练大语言模型获取地标-物体关系的实现方案，无需收集专用训练数据。\n我们提出的Prompter在ALFRED基准上，仅使用高级指令时准确率达41.53%，使用分步指令时达45.32%，分别较此前的最优水平显著提升5.46%和9.91%。
英文摘要：
Embodied Instruction Following (EIF) studies how autonomous mobile manipulation robots should be controlled to accomplish long-horizon tasks described by natural language instructions. While much research on EIF is conducted in simulators, the ultimate goal of the field is to deploy the agents in real life. This is one of the reasons why recent methods have moved away from training models end-to-end and take modular approaches, which do not need the costly expert operation data. However, as it is still in the early days of importing modular ideas to EIF, a search for modules effective in the EIF task is still far from a conclusion. In this paper, we propose to extend the modular design using knowledge obtained from two external sources. First, we show that embedding the physical constraints of the deployed robots into the module design is highly effective. Our design also allows the same modular system to work across robots of different configurations with minimal modifications. Second, we show that the landmark-based object search, previously implemented by a trained model requiring a dedicated set of data, can be replaced by an implementation that prompts pretrained large language models for landmark-object relationships, eliminating the need for collecting dedicated training data. Our proposed Prompter achieves 41.53\% and 45.32\% on the ALFRED benchmark with high-level instructions only and step-by-step instructions, respectively, significantly outperforming the previous state of the art by 5.46\% and 9.91\%.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
