Exploratory LQG Mean Field Games with Entropy Regularization

Dena Firoozi, Sebastian Jaimungal

备注：: To appear in Automatica

英文摘要：

We study a general class of entropy-regularized multi-variate LQG mean field games (MFGs) in continuous time with $K$ distinct sub-population of agents. We extend the notion of actions to action distributions (exploratory actions), and explicitly derive the optimal action distributions for individual agents in the limiting MFG. We demonstrate that the optimal set of action distributions yields an $ε$-Nash equilibrium for the finite-population entropy-regularized MFG. Furthermore, we compare the resulting solutions with those of classical LQG MFGs and establish the equivalence of their existence.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

阅读原文

阅读 PDF ↗

arXiv 页面 ↗HTML 原文 ↗

本文内容

英文摘要相关论文

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

继续探索

相关论文

LQG Online Learning

Optimal control theory and machine learning techniques are combined to formulate and solve in closed form an optimal control formulation of online learning from supervised examples with regularization of the updates. The connections with the classical Linear Quadratic Gaussian (LQG) optimal control problem, of which the proposed learning paradigm is a non-trivial variation as it involves random matrices, are investigated. The obtained optimal solutions are compared with the Kalman-filter estimate of the parameter vector to be learned. It is shown that the proposed algorithm is less sensitive t…

2016-12-14math.OC

离散时间线性指数二次高斯（LEQG）问题的探索性随机化

Exploratory Randomization for Discrete-Time Linear Exponential Quadratic Gaussian (LEQG) Problem

本文针对离散时间扩展LEQG控制问题引入控制随机化以实现探索，并利用自由能与相对熵对偶将其转化为带熵正则化的风险中性LQG问题，进而提出基于动态规划的求解方法，为随机化控制中熵正则化的使用提供了理论依据。

2025-09-18math.OCq-fin.PM

连续时间带共同噪声的探索性线性二次平均场控制问题中策略梯度学习算法的完整误差分析

Full error analysis of policy gradient learning algorithms for exploratory linear quadratic mean-field control problem in continuous time with common noise

针对带共同噪声的连续时间LQ平均场控制，本文分析PG强化学习算法，证明基于模型的收敛性及无模型两点梯度估计的全局线性收敛与样本复杂度，并用数值实验验证。

2024-08-05math.OCstat.ML

带熵正则化的倒向随机控制系统

Backward Stochastic Control System with Entropy Regularization

本文研究由倒向随机微分方程生成并带熵正则化的最优探索性控制问题，通过凸变分建立随机最大值原理，证明最优控制充分条件与隐式形式，并利用解耦技巧证明倒向线性二次情形下最优控制的存在唯一性。

2024-11-20math.OC

时间不一致平均场博弈的均衡：熵正则化的系统分析

Equilibrium for Time-inconsistent Mean Field Games: A Systematic Analysis by Entropy Regularization

University of Michigan（密歇根大学）；Zhongtai Securities Institute for Financial Studies, Shandong University（山东大学中泰证券金融研究院）；The Hong Kong Polytechnic University（香港理工大学）；Tsinghua University（清华大学）

针对时间不一致平均场博弈问题，提出熵正则化方法处理非局部均衡HJB系统，证明正则化均衡的存在性及收敛到原均衡，并给出策略迭代算法。

2026-05-28math.OC

时间不一致性下的均衡：通过消失熵正则化建立的新存在理论

Equilibrium under Time-Inconsistency: A New Existence Theory by Vanishing Entropy Regularization

Zhongtai Securities Institute for Financial Studies, Shandong University（山东大学中泰金融研究院）；Department of Applied Mathematics, The Hong Kong Polytechnic University（香港理工大学应用数学系）；China School of Banking and Finance, University of International Business and Economics（对外经济贸易大学中国金融学院）；School of Mathematics and Statistics, University of Sydney（悉尼大学数学与统计学院）

本文通过消失熵正则化方法，解决了时间不一致随机控制问题中均衡哈密顿-雅可比-贝尔曼方程解的存在性问题，无需依赖传统严格正则性假设。

2026-04-04math.APmath.OCmath.PR

更多相关论文收起更多论文

Exploration versus exploitation in reinforcement learning: a stochastic control approach

Columbia University（哥伦比亚大学）；The University of Texas at Austin（得克萨斯大学奥斯汀分校）；University of Oxford（牛津大学）

We consider reinforcement learning (RL) in continuous time and study the problem of achieving the best trade-off between exploration of a black box environment and exploitation of current knowledge. We propose an entropy-regularized reward function involving the differential entropy of the distributions of actions, and motivate and devise an exploratory formulation for the feature dynamics that captures repetitive learning under exploration. The resulting optimization problem is a revitalization of the classical relaxed stochastic control. We carry out a complete analysis of the problem in the…

2019-02-13cs.LGmath.OC