arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2310.01951cs.LGcs.AI
收藏

贝叶斯神经网络的概率可达-规避分析

Probabilistic Reach-Avoid for Bayesian Neural Networks

University of Oxford(牛津大学);TU Delft(代尔夫特理工大学);King’s College London(伦敦国王学院)

Matthew Wicker, Luca Laurenti, Andrea Patane, Nicola Paoletti, Alessandro Abate, Marta Kwiatkowska

AI总结:

针对基于模型强化学习中策略的安全认证需求,提出基于区间传播与反向递归的BNN可达-规避概率下界计算方法及最优控制合成算法,可显著提升可认证状态数与保证概率。

摘要:

基于模型的强化学习旨在同时学习未知随机环境的动力学,并合成在该环境中执行动作的最优策略。对于面向安全关键场景的策略而言,确保其在这类环境中做出的序列决策的安全性与鲁棒性是一项核心挑战。本研究探讨了两个互补的问题:第一,针对由贝叶斯神经网络(Bayesian neural network, BNN)描述动力学的动态模型,计算其迭代预测的可达-规避概率;第二,合成相对于给定可达-规避规范(即到达“目标”状态,同时避开一组“不安全”状态)和已学习BNN模型的最优控制策略。我们的方案利用区间传播和反向递归技术,计算策略的动作序列满足可达-规避规范的概率下界。这类计算得到的下界可为给定策略和BNN模型提供安全认证。随后我们提出了控制合成算法,以推导最大化上述安全概率下界的策略。我们在一系列以已学习BNN动力学模型为特征的控制基准上验证了方法的有效性。在我们最具挑战性的基准测试中,与纯数据驱动的策略相比,该最优合成算法可认证状态的数量提升了四倍以上,平均保证可达-规避概率提升了三倍以上。

英文摘要:

Model-based reinforcement learning seeks to simultaneously learn the dynamics of an unknown stochastic environment and synthesise an optimal policy for acting in it. Ensuring the safety and robustness of sequential decisions made through a policy in such an environment is a key challenge for policies intended for safety-critical scenarios. In this work, we investigate two complementary problems: first, computing reach-avoid probabilities for iterative predictions made with dynamical models, with dynamics described by Bayesian neural network (BNN); second, synthesising control policies that are optimal with respect to a given reach-avoid specification (reaching a "target" state, while avoiding a set of "unsafe" states) and a learned BNN model. Our solution leverages interval propagation and backward recursion techniques to compute lower bounds for the probability that a policy's sequence of actions leads to satisfying the reach-avoid specification. Such computed lower bounds provide safety certification for the given policy and BNN model. We then introduce control synthesis algorithms to derive policies maximizing said lower bounds on the safety probability. We demonstrate the effectiveness of our method on a series of control benchmarks characterized by learned BNN dynamics models. On our most challenging benchmark, compared to purely data-driven policies the optimal synthesis algorithm is able to provide more than a four-fold increase in the number of certifiable states and more than a three-fold increase in the average guaranteed reach-avoid probability.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑