arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2112.06018cs.LGcs.SYeess.SYmath.OC
收藏

控制指导的强化学习:面向数据驱动和基于模型控制的集成

Control-Tutored Reinforcement Learning: Towards the Integration of Data-Driven and Model-Based Control

University of Naples Federico II(那不勒斯费德里科二世大学);Scuola Superiore Meridionale(南方高等研究院);University of Salerno(萨莱诺大学);University College London(伦敦大学学院);University of Bologna(博洛尼亚大学)

F. De Lellis, M. Coraggio, G. Russo, M. Musolesi, M. di Bernardo

摘要:

我们提出了一个架构,其中的反馈控制器上的近似模型的环境协助学习过程,以提高其数据效率。这种架构,我们称之为控制辅导Q学习(CTQL),有两种可选的风格。前者基于定义奖励函数,以便可以使用布尔条件来确定何时采用控制导师策略,而后者称为概率CTQL(pCTQL),而是基于在学习期间以一定概率执行对导师的调用。通过将OpenAI Gym中定义的倒立摆的稳定性视为代表性问题,这两种方法都得到了验证,并完全针对Q-Learning进行了基准测试。

英文摘要:

We present an architecture where a feedback controller derived on an approximate model of the environment assists the learning process to enhance its data efficiency. This architecture, which we term as Control-Tutored Q-learning (CTQL), is presented in two alternative flavours. The former is based on defining the reward function so that a Boolean condition can be used to determine when the control tutor policy is adopted, while the latter, termed as probabilistic CTQL (pCTQL), is instead based on executing calls to the tutor with a certain probability during learning. Both approaches are validated, and thoroughly benchmarked against Q-Learning, by considering the stabilization of an inverted pendulum as defined in OpenAI Gym as a representative problem.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑