arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2210.04964cs.ROcs.AIcs.CL
收藏

利用具备环境感知能力的语言模型生成可执行动作规划

Generating Executable Action Plans with Environmentally-Aware Language Models

University of North Carolina at Chapel Hill(北卡罗来纳大学教堂山分校)

Maitrey Gramopadhye, Daniel Szafir

AI总结:

针对大语言模型生成的机器人动作规划因未考虑环境而难以执行的问题,提出将环境物体与关系输入LLM、搭配新型评分函数的环境感知规划方法,在VirtualHome和ActivityPrograms上验证可大幅提升规划的可执行性与正确性。

摘要:

基于海量文本数据集训练的大语言模型(LLM)近期展现出从高层文本查询为机器人生成动作规划的潜力。但这类模型通常未考虑机器人所处的环境,由于规划动作存在歧义或受环境约束,生成的规划可能实际无法执行。本文提出一种生成具备环境感知的动作规划的方法,能提升智能体的可执行性。我们的方法将环境中的物体及物体间关系作为额外输入整合进LLM的动作规划生成流程,让系统具备对周围环境的感知能力,生成的规划中每个动作都能对应到场景中存在的物体。我们还设计了一种新颖的评分函数,该函数在生成动作步骤并将其与物体关联的同时,还能帮助系统区分不同物体实例并考量其状态。我们使用VirtualHome模拟器和ActivityPrograms知识库对所提方法进行了评估,结果表明,与已有研究相比,我们的系统生成的动作规划可执行性提升了310%,正确性提升了147%。完整代码和方法演示已公开于https://github.com/hri-ironlab/scene_aware_language_planner。

英文摘要:

Large Language Models (LLMs) trained using massive text datasets have recently shown promise in generating action plans for robotic agents from high level text queries. However, these models typically do not consider the robot's environment, resulting in generated plans that may not actually be executable, due to ambiguities in the planned actions or environmental constraints. In this paper, we propose an approach to generate environmentally-aware action plans that agents are better able to execute. Our approach involves integrating environmental objects and object relations as additional inputs into LLM action plan generation to provide the system with an awareness of its surroundings, resulting in plans where each generated action is mapped to objects present in the scene. We also design a novel scoring function that, along with generating the action steps and associating them with objects, helps the system disambiguate among object instances and take into account their states. We evaluated our approach using the VirtualHome simulator and the ActivityPrograms knowledge base and found that action plans generated from our system had a 310% improvement in executability and a 147% improvement in correctness over prior work. The complete code and a demo of our method is publicly available at https://github.com/hri-ironlab/scene_aware_language_planner.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑