arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2403.05693cs.LG
收藏

复杂航天器任务分配的屏蔽深度强化学习

Shielded Deep Reinforcement Learning for Complex Spacecraft Tasking

University of Colorado Boulder(科罗拉多大学博尔德分校)

Robert Reed, Hanspeter Schaub, Morteza Lahijanian

AI总结:

本文用线性时序逻辑形式化航天器任务与安全要求,自动构建奖励函数,并提出三种带概率保证的屏蔽设计,实验验证了其与策略交互的灵活性。

摘要:

通过屏蔽深度强化学习(SDRL)实现的自主航天器控制已成为一个快速发展的研究领域。然而,屏蔽的构建和任务的定义仍然是非正式的,导致策略在安全性上没有保证,且强化学习智能体的目标模糊。在本文中,我们首先探索使用形式语言,即线性时序逻辑(LTL),来形式化航天器任务和安全要求。然后,我们定义了一种从co-safe LTL规范自动构建奖励函数的方法,以便在SDRL框架中进行有效训练。我们还研究了从安全LTL规范为航天器应用构建屏蔽的方法,并提出了三种提供概率保证的设计。通过多个实验,我们展示了这些屏蔽如何与不同策略交互,以及奖励结构的灵活性。

英文摘要:

Autonomous spacecraft control via Shielded Deep Reinforcement Learning (SDRL) has become a rapidly growing research area. However, the construction of shields and the definition of tasking remains informal, resulting in policies with no guarantees on safety and ambiguous goals for the RL agent. In this paper, we first explore the use of formal languages, namely Linear Temporal Logic (LTL), to formalize spacecraft tasks and safety requirements. We then define a manner in which to construct a reward function from a co-safe LTL specification automatically for effective training in SDRL framework. We also investigate methods for constructing a shield from a safe LTL specification for spacecraft applications and propose three designs that provide probabilistic guarantees. We show how these shields interact with different policies and the flexibility of the reward structure through several experiments.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑