任何房屋任何任务:为抽象人类任务的可扩展长周期规划
TGPO: Trace-Guided Policy Optimization for Robot Task Planning via Verifiable Subgoal Generation
AI总结:
AHAT是一种优化长周期规划的家庭任务规划器,通过结合LLM和TGPO算法,在复杂家庭任务中实现高效规划。
摘要:
开放世界语言条件任务规划对在大规模家庭环境中操作的机器人至关重要。尽管许多近期工作试图通过提示或训练来解决这个问题,但可扩展性仍然是一个关键挑战。性能往往随着环境大小、计划长度、指令模糊性和约束复杂性的增加而迅速下降。在本文中,我们提出了Any House Any Task(AHAT),一种家庭任务规划器,专门优化了在大型环境中进行长周期规划的能力,以应对模糊的人类指令。其核心是AHAT利用一个训练好的LLM,将任务指令和文本场景图映射到基于规划领域定义语言(PDDL)定义的 grounded subgoals。这些子目标随后通过显式的符号推理得到解决,以生成可行且最优的长周期计划。为了增强模型分解复杂和模糊意图的能力,我们引入了TGPO,一种新的强化学习算法,该算法将外部纠正中间推理轨迹整合到组相对策略优化(GRPO)中。实验表明,AHAT在最先进的提示、规划和学习方法上实现了显著的性能提升,特别是在具有简短指令但需要复杂执行计划的人类风格家庭任务中。
英文摘要:
Robot task planning in real-world environments requires mapping abstract natural language instructions to executable action sequences under long horizons and complex constraints. While large language models (LLMs) provide strong commonsense reasoning, they often fail to generate reliable and feasible plans. In contrast, symbolic planners ensure feasibility and optimality but require well-specified goals and cannot directly interpret high-level human intent.We formulate robot task planning as learning to generate verifiable subgoals in the Planning Domain Definition Language (PDDL), bridging language understanding and symbolic planning. To address the challenges of sparse and noisy supervision, we propose Trace-Guided Policy Optimization (TGPO), a reinforcement learning framework that improves structured subgoal generation through (i) verifier-grounded rewards, (ii) external correction of intermediate reasoning traces, and (iii) constrained policy updates that incorporate corrected traces into training.We evaluate TGPO on large-scale household planning tasks with long horizons, abstract instructions, and complex constraints. TGPO significantly outperforms prompting-based and reinforcement learning baselines, with the largest gains on abstract tasks. Furthermore, TGPO integrates naturally with symbolic planners and language-conditioned executors, enabling robust long-horizon planning and combinatorial generalization in realistic environments.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。