arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2402.16342cs.AIcs.RO
收藏

面向太空任务的基于双层马尔可夫决策过程的应急规划

Contingency Planning Using Bi-level Markov Decision Processes for Space Missions

Stanford University(斯坦福大学);NASA Ames Research Center(美国国家航空航天局艾姆斯研究中心)

Somrita Banerjee, Edward Balaban, Mark Shirley, Kevin Bradner, Marco Pavone

AI总结:

本文提出双层 MDP 框架,用于解决行星巡视器任务中因动作空间大和时间范围长导致的计算难题,并在 RoverGridWorld 上验证了其计算可处理性与近最优策略。

摘要:

本工作聚焦于科学任务的自主应急规划,通过在发生延迟或偏离名义任务计划时,从状态空间中任意非标称点实现快速策略计算。成功的应急规划涉及在随机场景中管理通常与动作概率相关的风险与回报。马尔可夫决策过程(MDPs)被用于对此类场景中的决策进行数学建模。然而,在行星巡视器行进规划这一特定情形下,庞大的动作空间和较长的规划时间范围带来了计算挑战。本文提出一种双层 MDP 框架,以提高计算可处理性,同时与现有任务规划实践保持一致,并增强 AI 驱动解决方案的可解释性与可信度。我们讨论了将任务规划 MDP 转换为双层 MDP 的方法,并在 RoverGridWorld(一个用于巡视器任务规划的改进版 GridWorld 环境)上对该框架进行了测试。我们证明了双层 MDP 方法可实现的计算可处理性和近最优策略,并强调了随着问题复杂度增长,计算时间与策略最优性之间的权衡。本工作有助于在科学任务背景下实现更高效、更灵活的应急规划。

英文摘要:

This work focuses on autonomous contingency planning for scientific missions by enabling rapid policy computation from any off-nominal point in the state space in the event of a delay or deviation from the nominal mission plan. Successful contingency planning involves managing risks and rewards, often probabilistically associated with actions, in stochastic scenarios. Markov Decision Processes (MDPs) are used to mathematically model decision-making in such scenarios. However, in the specific case of planetary rover traverse planning, the vast action space and long planning time horizon pose computational challenges. A bi-level MDP framework is proposed to improve computational tractability, while also aligning with existing mission planning practices and enhancing explainability and trustworthiness of AI-driven solutions. We discuss the conversion of a mission planning MDP into a bi-level MDP, and test the framework on RoverGridWorld, a modified GridWorld environment for rover mission planning. We demonstrate the computational tractability and near-optimal policies achievable with the bi-level MDP approach, highlighting the trade-offs between compute time and policy optimality as the problem's complexity grows. This work facilitates more efficient and flexible contingency planning in the context of scientific missions.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑