通过专家示范学习在未知约束下的安全维护：从Q学习视角出发
Learning to maintain safety through expert demonstrations in settings with unknown constraints: A Q-learning perspective
AI总结：
本文提出SafeQIL算法，通过专家示范在未知约束环境下学习安全策略，结合Q值评估和安全考量，提升高奖励轨迹的可能性。
摘要：
给定一组展示在具有可观测奖励但未知约束的受限马尔可夫决策过程（MDP）中安全执行任务的轨迹，我们旨在找到一个政策，最大化演示轨迹在保守性和显著增加高奖励轨迹可能性之间的平衡，但可能包含不安全步骤。为了实现这些目标，我们旨在学习一个最大化最有前途的轨迹概率的政策，相对于演示。为此，我们将个体状态-动作对的“前途”以Q值来表述，这些Q值取决于任务特定的奖励以及对状态安全性的评估，混合了奖励和安全性的期望。这导致了在约束下的逆学习问题的“安全Q学习”视角：所设计的Safe Q Inverse Constrained Reinforcement Learning（SafeQIL）算法与最先进的逆约束强化学习算法在一系列具有挑战性的基准任务上进行了比较，展示了其优势。
英文摘要：
Given a set of trajectories demonstrating the execution of a task safely in a constrained MDP with observable rewards but with unknown constraints and non-observable costs, we aim to find a policy that maximizes the likelihood of demonstrated trajectories trading the balance between being conservative and increasing significantly the likelihood of high-rewarding trajectories but with potentially unsafe steps. Having these objectives, we aim towards learning a policy that maximizes the probability of the most $promising$ trajectories with respect to the demonstrations. In so doing, we formulate the ``promise" of individual state-action pairs in terms of $Q$ values, which depend on task-specific rewards as well as on the assessment of states' safety, mixing expectations in terms of rewards and safety. This entails a safe Q-learning perspective of the inverse learning problem under constraints: The devised Safe $Q$ Inverse Constrained Reinforcement Learning (SafeQIL) algorithm is compared to state-of-the art inverse constraint reinforcement learning algorithms to a set of challenging benchmark tasks, showing its merits.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
