强化学习中时态逻辑约束的概率保证满足
Probabilistically Guaranteed Satisfaction of Temporal Logic Constraints During Reinforcement Learning
摘要：
我们提出了一种新的约束强化学习方法，在马尔可夫决策过程中找到最优的政策，同时满足时间逻辑约束，在整个学习过程中所需的概率。提出了一种自动机理论的方法，以确保概率满足的约束，在每一集，这是不同的惩罚违规，以达到约束满足后，足够大的情节。所提出的方法是基于计算约束满足的概率的下限，并根据需要调整探索行为。我们提出的概率约束满足所提出的方法实现的理论结果。我们还在无人机场景中数值演示了所提出的想法，其中约束是执行定期到达的拾取和交付任务，目标是飞越高回报区域，同时执行空中监测。
英文摘要：
We propose a novel constrained reinforcement learning method for finding optimal policies in Markov Decision Processes while satisfying temporal logic constraints with a desired probability throughout the learning process. An automata-theoretic approach is proposed to ensure the probabilistic satisfaction of the constraint in each episode, which is different from penalizing violations to achieve constraint satisfaction after a sufficiently large number of episodes. The proposed approach is based on computing a lower bound on the probability of constraint satisfaction and adjusting the exploration behavior as needed. We present theoretical results on the probabilistic constraint satisfaction achieved by the proposed approach. We also numerically demonstrate the proposed idea in a drone scenario, where the constraint is to perform periodically arriving pick-up and delivery tasks and the objective is to fly over high-reward zones to simultaneously perform aerial monitoring.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
