arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 1912.07883math.OCmath.PR
收藏

具有共同噪声和开环控制的平均场马氏决策过程

Mean-field Markov decision processes with common noise and open-loop controls

Médéric Motte, Huyên Pham

摘要:

本文详尽地研究了马尔可夫决策过程(MDP)在平均场相互作用下的状态和行为,以及在有限时域上对开环控制进行优化时的状态和行为。这样的模型,称为CMKV-MDP的条件McKean-Vlasov MDP,出现,并在这里得到严格的收敛速率作为渐近问题的N-合作代理人控制的社会规划者/影响者,观察环境噪声,但不一定是个体状态的代理人。我们强调了放松控制和随机化假设对这类模型相对于经典MDP理论的重要作用。证明了CMKV-MDP与一般提升MDP在概率测度空间上的对应性,建立了值函数所满足的动态规划Bellman不动点方程,以及最优随机反馈控制的存在性.证明的论点涉及一个原始的可测量的最佳耦合的Wasserstein距离。这提供了一个程序,在一个大的人口互动协作代理的学习策略。MSC分类:90 C40、49 L20。

英文摘要:

We develop an exhaustive study of Markov decision process (MDP) under mean field interaction both on states and actions in the presence of common noise, and when optimization is performed over open-loop controls on infinite horizon. Such model, called CMKV-MDP for conditional McKean-Vlasov MDP, arises and is obtained here rigorously with a rate of convergence as the asymptotic problem of N-cooperative agents controlled by a social planner/influencer that observes the environment noises but not necessarily the individual states of the agents. We highlight the crucial role of relaxed controls and randomization hypothesis for this class of models with respect to classical MDP theory. We prove the correspondence between CMKV-MDP and a general lifted MDP on the space of probability measures, and establish the dynamic programming Bellman fixed point equation satisfied by the value function, as well as the existence of-optimal randomized feedback controls. The arguments of proof involve an original measurable optimal coupling for the Wasserstein distance. This provides a procedure for learning strategies in a large population of interacting collaborative agents. MSC Classification: 90C40, 49L20.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑