arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2310.01538eess.SYcs.SY
收藏

面向安全强化学习的风险敏感型抑制控制

Risk-Sensitive Inhibitory Control for Safe Reinforcement Learning

Armin Lederer, Erfaun Noorani, John S. Baras, Sandra Hirche

AI总结:

受人类反应抑制与风险态度相关的研究启发,提出基于风险感知价值函数条件的风险敏感型抑制控制方法,通过强化学习学习价值函数并在线执行安全约束,在仿真中验证了该安全强化学习控制方案的有效性。

摘要:

人类具备在必要时偏离固有行为的能力,这一认知过程被称为反应抑制。近年来,类似的方法因可保障控制安全性,独立受到了越来越多的关注。这类方法通过控制屏障函数或预测安全滤波器实现,能够对标称控制律(例如通过强化学习得到的控制律)进行在线调整,从而有效确保状态约束得到满足。尽管这些抑制控制实现方案的研究重点一直是风险中性公式,但人类研究已表明反应抑制与风险态度之间存在紧密联系。受这一发现启发,我们提出了一种灵活的风险敏感型抑制控制方法。该方法基于价值函数的风险感知条件,可保证状态约束的满足性。我们提出了一种利用强化学习常用技术学习这些价值函数的方法,并推导了该方法成功的充分条件。通过利用学习到的价值函数在线强制执行推导得到的安全条件,可有效实现风险敏感型抑制控制。仿真实验验证了所提出控制方案的有效性。

英文摘要:

Humans have the ability to deviate from their natural behavior when necessary, which is a cognitive process called response inhibition. Similar approaches have independently received increasing attention in recent years for ensuring the safety of control. Realized using control barrier functions or predictive safety filters, these approaches can effectively ensure the satisfaction of state constraints through an online adaptation of nominal control laws, e.g., obtained through reinforcement learning. While the focus of these realizations of inhibitory control has been on risk-neutral formulations, human studies have shown a tight link between response inhibition and risk attitude. Inspired by this insight, we propose a flexible, risk-sensitive method for inhibitory control. Our method is based on a risk-aware condition for value functions, which guarantees the satisfaction of state constraints. We propose a method for learning these value functions using common techniques from reinforcement learning and derive sufficient conditions for its success. By enforcing the derived safety conditions online using the learned value function, risk-sensitive inhibitory control is effectively achieved. The effectiveness of the developed control scheme is demonstrated in simulations.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑