可学习的安全措施
A Learnable Safety Measure
摘要:
失败对于学习控制物理系统是具有挑战性的,因为它们有损坏的风险,耗时的重置,并且通常提供很少的梯度信息。为勘探增加安全约束通常需要大量的先验知识和领域专业知识。我们提出了一个安全措施,它隐含地捕捉系统动态与一组故障状态。该测度不仅可以作为安全函数,而且可以直接计算安全状态-动作对的集合。此外,我们展示了一种无模型的方法来学习这个措施,使用高斯过程的主动采样。虽然只有在学习安全措施后才能保证安全,但我们表明,通过在学习过程中使用估计措施,故障已经可以大大减少。
英文摘要:
Failures are challenging for learning to control physical systems since they risk damage, time-consuming resets, and often provide little gradient information. Adding safety constraints to exploration typically requires a lot of prior knowledge and domain expertise. We present a safety measure which implicitly captures how the system dynamics relate to a set of failure states. Not only can this measure be used as a safety function, but also to directly compute the set of safe state-action pairs. Further, we show a model-free approach to learn this measure by active sampling using Gaussian processes. While safety can only be guaranteed after learning the safety measure, we show that failures can already be greatly reduced by using the estimated measure during learning.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。