保证学习的自主性：一个元认知强化学习框架
Assured Learning-enabled Autonomy: A Metacognitive Reinforcement Learning Framework
摘要：
具有预先指定的奖励函数的强化学习（RL）代理不能在不确定系统可能遇到的各种情况下提供有保证的安全性。为了保证性能，同时确保满足各种情况下的安全约束，本文提出了一个有保证的自主控制框架，通过授权RL算法与元认知学习能力。更具体地说，自适应的RL代理的奖励函数参数进行元认知决策层，以确保RL代理的可行性。也就是说，确保RL代理学习的策略满足信号时序逻辑指定的安全约束，同时实现尽可能多的性能。元认知层监控RL代理的行为下任何可能的未来安全违规，并采用高层贝叶斯RL算法主动适应较低层RL代理的奖励函数。为了最小化高层贝叶斯RL干预，元认知层利用适应度函数作为度量来评估低层RL代理在满足安全性和活性规范方面的成功，并且仅当存在低层RL失败的风险时，高层贝叶斯RL才进行干预。最后通过仿真实例验证了该方法的有效性。
英文摘要：
Reinforcement learning (RL) agents with pre-specified reward functions cannot provide guaranteed safety across variety of circumstances that an uncertain system might encounter. To guarantee performance while assuring satisfaction of safety constraints across variety of circumstances, an assured autonomous control framework is presented in this paper by empowering RL algorithms with metacognitive learning capabilities. More specifically, adapting the reward function parameters of the RL agent is performed in a metacognitive decision-making layer to assure the feasibility of RL agent. That is, to assure that the learned policy by the RL agent satisfies safety constraints specified by signal temporal logic while achieving as much performance as possible. The metacognitive layer monitors any possible future safety violation under the actions of the RL agent and employs a higher-layer Bayesian RL algorithm to proactively adapt the reward function for the lower-layer RL agent. To minimize the higher-layer Bayesian RL intervention, a fitness function is leveraged by the metacognitive layer as a metric to evaluate success of the lower-layer RL agent in satisfaction of safety and liveness specifications, and the higher-layer Bayesian RL intervenes only if there is a risk of lower-layer RL failure. Finally, a simulation example is provided to validate the effectiveness of the proposed approach.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
