一图胜千言：像素规划的语言模型
A Picture is Worth a Thousand Words: Language Models Plan from Pixels
摘要：
规划是人工智能体在现实环境中执行长期任务的重要能力。在这项工作中，我们探讨了使用预训练的语言模型（PLM）的原因计划序列的文本指令在具体的视觉环境。现有的基于PLM的规划方法或者假设观察结果以文本的形式可用（例如，由字幕模型提供）、仅从指令推理计划、或者以有限的方式（诸如预先训练的示能表示功能）并入关于视觉环境的信息。相比之下，我们表明，PLM可以准确地计划，即使观察直接编码为输入提示的PLM。我们表明，这种简单的方法优于以前的方法在ALFWorld和VirtualHome基准测试的实验。
英文摘要：
Planning is an important capability of artificial agents that perform long-horizon tasks in real-world environments. In this work, we explore the use of pre-trained language models (PLMs) to reason about plan sequences from text instructions in embodied visual environments. Prior PLM based approaches for planning either assume observations are available in the form of text (e.g., provided by a captioning model), reason about plans from the instruction alone, or incorporate information about the visual environment in limited ways (such as a pre-trained affordance function). In contrast, we show that PLMs can accurately plan even when observations are directly encoded as input prompts for the PLM. We show that this simple approach outperforms prior approaches in experiments on the ALFWorld and VirtualHome benchmarks.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
