基于技能的安全强化学习与风险规划
Skill-based Safe Reinforcement Learning with Risk Planning
AI总结：
本文提出安全技能规划（SSkP）方法，利用离线演示数据通过PU学习构建技能风险预测器，并结合在线风险规划高效学习风险规避的安全策略，在多个机器人仿真基准中优于现有安全强化学习方法。
摘要：
安全强化学习（Safe RL）旨在确保强化学习智能体在与真实世界环境交互学习时的安全性，因为不当动作可能导致高昂代价或严重后果。本文提出了一种新颖的安全技能规划（SSkP）方法，通过利用辅助离线演示数据来增强有效的安全强化学习。SSkP包含一个两阶段过程。首先，我们采用PU学习从离线演示数据中学习技能风险预测器。然后，基于学习到的技能风险预测器，我们开发了一种新颖的风险规划过程，以增强在线安全强化学习，并通过与在线强化学习环境交互高效地学习风险规避的安全策略，同时使技能风险预测器适应环境。我们在多个基准机器人仿真环境中进行了实验。实验结果表明，所提出的方法持续优于先前最先进的安全强化学习方法。
英文摘要：
Safe Reinforcement Learning (Safe RL) aims to ensure safety when an RL agent conducts learning by interacting with real-world environments where improper actions can induce high costs or lead to severe consequences. In this paper, we propose a novel Safe Skill Planning (SSkP) approach to enhance effective safe RL by exploiting auxiliary offline demonstration data. SSkP involves a two-stage process. First, we employ PU learning to learn a skill risk predictor from the offline demonstration data. Then, based on the learned skill risk predictor, we develop a novel risk planning process to enhance online safe RL and learn a risk-averse safe policy efficiently through interactions with the online RL environment, while simultaneously adapting the skill risk predictor to the environment. We conduct experiments in several benchmark robotic simulation environments. The experimental results demonstrate that the proposed approach consistently outperforms previous state-of-the-art safe RL methods.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
