arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2303.06993stat.MLmath.OC
收藏

连续时间平均场控制的Actor-Critical学习

Actor-Critic learning for mean-field control in continuous time

Noufel Frikha, Maximilien Germain, Mathieu Laurière, Huyên Pham, Xuanye Song

摘要:

我们研究了强化学习环境中连续时间平均场控制的策略梯度。通过考虑具有熵正则化的随机政策,我们推导出价值函数的梯度期望表示,该表示适用于行动者-批评者类型算法,其中价值函数和政策根据状态的观察样本和人口状态分布的无模型估计交替学习,无论是通过离线还是在线学习。在线性二次平均场的框架下,我们得到了一个精确的参数化的演员和评论家的功能上定义的Wasserstein空间。最后,我们用一些具体的数值实验来说明我们的算法的结果。

英文摘要:

We study policy gradient for mean-field control in continuous time in a reinforcement learning setting. By considering randomised policies with entropy regularisation, we derive a gradient expectation representation of the value function, which is amenable to actor-critic type algorithms, where the value functions and the policies are learnt alternately based on observation samples of the state and model-free estimation of the population state distribution, either by offline or online learning. In the linear-quadratic mean-field framework, we obtain an exact parametrisation of the actor and critic functions defined on the Wasserstein space. Finally, we illustrate the results of our algorithms with some numerical experiments on concrete examples.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑