arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2412.08565cs.LGcs.AI
收藏

GenPlan:作为自适应规划器的生成式序列模型

GenPlan: Generative Sequence Models as Adaptive Planners

University of Waterloo(滑铁卢大学)

Akash Karthikeyan, Yash Vardhan Pant

AI总结:

GenPlan 利用离散流模型和迭代去噪生成目标与动作序列,实现多任务自适应规划,在自适应规划任务上比现有最优方法性能提升超过 10%。

摘要:

序列模型通过利用先前收集的示范,在行为规划中已展现出显著成功。然而,解决多任务使命仍然是一项重大挑战,尤其是当规划器必须适应未见过的约束和任务时,例如发现目标和解锁门。此类行为规划问题之所以难以解决,原因在于:a) 智能体无法适应超出其奖励函数所学单一任务之外的情况;b) 当仅在平面环境中训练时,无法泛化到新环境,例如带有墙壁和锁门的场景。因此,最先进的决策方法仅限于那些所需任务在训练示范中得到了充分表示、并且能够在较短(时间)规划视野内解决的使命。为解决这一问题,我们提出了 GenPlan:一种随机且自适应的规划器,它利用离散流模型进行生成式序列建模,从而实现样本高效的探索与利用。该框架依赖迭代去噪过程来生成目标和动作序列。该方法能够捕捉多模态动作分布,并促进目标和任务发现,从而泛化到分布外任务和环境,即不属于训练数据的使命。我们通过多个仿真环境证明了所提方法的有效性。值得注意的是,GenPlan 在自适应规划任务上比最先进方法性能高出超过 10%,在这些任务中,智能体在利用单目标到达任务的示范的同时适应多任务使命。我们的代码可在 https://github.com/CL2-UWaterloo/GenPlan 获取。

英文摘要:

Sequence models have demonstrated remarkable success in behavioral planning by leveraging previously collected demonstrations. However, solving multi-task missions remains a significant challenge, particularly when the planner must adapt to unseen constraints and tasks, such as discovering goals and unlocking doors. Such behavioral planning problems are challenging to solve due to: a) agents failing to adapt beyond the single task learned through their reward function, and b) inability to generalize to new environments, e.g., those with walls and locked doors, when trained only in planar environments. Consequently, state-of-the-art decision-making methods are limited to missions where the required tasks are well-represented in the training demonstrations and can be solved within a short (temporal) planning horizon. To address this, we propose GenPlan: a stochastic and adaptive planner that leverages discrete-flow models for generative sequence modeling, enabling sample-efficient exploration and exploitation. This framework relies on an iterative denoising procedure to generate a sequence of goals and actions. This approach captures multi-modal action distributions and facilitates goal and task discovery, thereby generalizing to out-of-distribution tasks and environments, i.e., missions not part of the training data. We demonstrate the effectiveness of our method through multiple simulation environments. Notably, GenPlan outperforms state-of-the-art methods by over 10% on adaptive planning tasks, where the agent adapts to multi-task missions while leveraging demonstrations from single-goal-reaching tasks. Our code is available at https://github.com/CL2-UWaterloo/GenPlan.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑