arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2410.02898eess.SYcs.LGcs.ROcs.SY
收藏

用于可达-规避-保持问题的深度强化学习

Deep Reinforcement Learning for Reach-Avoid-Stay Problems

North Carolina State University(北卡罗来纳州立大学);University of California, Berkeley(加利福尼亚大学伯克利分校);Honda Research Institute(本田研究所)

Gabriel Chenevert, Jingqi Li, Achyuta kannan, Sangjae Bae, Donggun Lee

AI总结:

本文提出两步深度强化学习框架,联合学习最大鲁棒RAS集与对应控制策略,仿真显示该框架可精确计算RAS集且性能优于基线方法。

摘要:

可达-规避-保持(RAS)任务在系统必须在所有有界扰动下安全到达目标集并保持在其中的应用中至关重要。现有方法要么难以计算最大鲁棒RAS集(所有可完成RAS任务的状态集合),要么在处理一般动态系统时存在局限。为应对这些挑战,本文提出一种两步深度强化学习框架,联合学习最大鲁棒RAS集与对应的控制策略。第一步在目标集内识别最大鲁棒控制不变集,并推导确保系统保持在其中的策略;第二步以该不变集为目标计算最大鲁棒可达-规避(RA)集,且已证明此RA集等价于最大鲁棒RAS集。利用该结果,从两个分步策略构建切换策略,该策略构成保证完成RAS任务的有效策略。仿真结果表明,所提框架:1)在无训练误差时计算出精确的最大鲁棒RAS集,得到限制最少的RAS策略;2)以高准确率识别RAS集,且在RAS任务上优于基线方法。

英文摘要:

Reach-Avoid-Stay (RAS) tasks are essential in applications where systems must safely reach a target set and remain within it under all bounded disturbances. Existing approaches either struggle to compute the maximal robust RAS set, the set of all states from which the RAS task is achievable, or are limited in handling general dynamic systems. To address these challenges, this paper proposes a two-step deep reinforcement learning framework that jointly learns the maximal robust RAS set and the corresponding control policy. The first step identifies the maximal robust control-invariant set within the target set and derives a policy that ensures the system remains within it. The second step computes the maximal robust reach-avoid (RA) set using this invariant set as the target, and it is proven that this RA set is equivalent to the maximal robust RAS set. Leveraging this result, a switching policy is constructed from the two step-wise policies, which constitutes a valid policy guaranteeing completion of the RAS task. Simulation results demonstrate that the proposed framework (1) computes the exact maximal robust RAS set in the absence of training errors, yielding the least restrictive RAS policy, and (2) identifies the RAS set with high accuracy while outperforming baseline methods on RAS tasks.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑