程序即规程:通过自然语言对情境化智能体进行分层控制
Procedures as Programs: Hierarchical Control of Situated Agents through Natural Language
AI总结:
本文提出将规程表示为程序的分层形式化方法,并设计由规划器和反应器组成的分层模块化网络,在IQA和ALFRED数据集上显著超越反应式基线,同时具备更高的数据效率和快速迭代能力。
摘要:
当人类构思如何执行特定任务时,他们会以分层方式进行:将更高层级的任务拆分为更小的子任务。然而,在关于情境化智能体的自然语言(NL)指令的文献中,大多数工作将待执行的规程视为简单动作的扁平序列,或者任何规程层级充其量都是浅层的。在本文中,我们提出了一种将规程表示为程序的形式化方法,这是一种强大而直观的方法,用于表示智能体指挥与控制的分层规程知识。我们进一步提出了一种分层模块化网络的建模范式,该范式由一个规划器和多个反应器组成,规划器和反应器将自然语言意图转换为可执行程序的预测,并探测环境以获取完成程序执行所需的信息。我们在IQA和ALFRED数据集上实例化了该框架,用于自然语言指令跟随。我们的模型在这两个数据集上均以较大优势优于反应式基线。我们还证明了我们的框架具有更高的数据效率,并且允许快速的迭代开发。
英文摘要:
When humans conceive how to perform a particular task, they do so hierarchically: splitting higher-level tasks into smaller sub-tasks. However, in the literature on natural language (NL) command of situated agents, most works have treated the procedures to be executed as flat sequences of simple actions, or any hierarchies of procedures have been shallow at best. In this paper, we propose a formalism of procedures as programs, a powerful yet intuitive method of representing hierarchical procedural knowledge for agent command and control. We further propose a modeling paradigm of hierarchical modular networks, which consist of a planner and reactors that convert NL intents to predictions of executable programs and probe the environment for information necessary to complete the program execution. We instantiate this framework on the IQA and ALFRED datasets for NL instruction following. Our model outperforms reactive baselines by a large margin on both datasets. We also demonstrate that our framework is more data-efficient, and that it allows for fast iterative development.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。