Game theory has been an effective tool in the control of disease spread and in suggesting optimal policies at both individual and area levels. In this paper, we propose a multi-region SEIR model based on stochastic differential game theory, aiming to formulate optimal regional policies for infectious diseases. Specifically, we enhance the standard epidemic SEIR model by taking into account the social and health policies issued by multiple region planners. This enhancement makes the model more realistic and powerful. However, it also introduces a formidable computational challenge due to the high dimensionality of the solution space brought by the presence of multiple regions. This significant numerical difficulty of the model structure motivates us to generalize the deep fictitious algorithm introduced in [Han and Hu, MSML2020, pp.221--245, PMLR, 2020] and develop an improved algorithm to overcome the curse of dimensionality. We apply the proposed model and algorithm to study the COVID-19 pandemic in three states: New York, New Jersey, and Pennsylvania. The model parameters are estimated from real data posted by the Centers for Disease Control and Prevention (CDC). We are able to show the effects of the lockdown/travel ban policy on the spread of COVID-19 for each state and how their policies affect each other.
University of California, Santa Barbara（加州大学圣巴巴拉分校）；Flatiron Institute（熨斗研究院）
Game theory has been an effective tool in the control of disease spread and in suggesting optimal policies at both individual and area levels. In this AMS Notices article, we focus on the decision-making development for the intervention of COVID-19, aiming to provide mathematical models and efficient machine learning methods, and justifications for related policies that have been implemented in the past and explain how the authorities' decisions affect their neighboring regions from a game theory viewpoint.
In this research paper we modify a classical SIR model to better adapt to the dynamics of COVID-19, that is we propose the heterogeneous SQAIRD model where COVID-19 spreads over a population of economic agents, namely: the elderly, adults and young people. We then compute and simulate an optimal control problem faced by a Government, where its objective is to minimize the costs generated by the pandemics using as control a compulsory quarantine measure (that is, a lockdown). We first analyze the problem from a theoretical perspective, claiming that different lockdown policies (total lockdown, …
Governments across the world are currently facing the task of selecting suitable intervention strategies to cope with the effects of the COVID-19 pandemic. This is a highly challenging task, since harsh measures may result in economic collapse while a relaxed strategy might lead to a high death toll. Motivated by this, we consider the problem of forming intervention strategies to mitigate the impact of the COVID-19 pandemic that optimize the trade-off between the number of deceases and the socio-economic costs. We demonstrate that the healthcare capacity and the testing rate highly affect the …
We consider time-homogeneous uniformly nondegenerate stochastic differential games in domains and propose constructing $\varepsilon$-optimal strategies and policies by using adjoint Markov strategies and adjoint Markov policies which are actually time-homogeneous Markov, however, relative not to the original process but to a couple of processes governed by a system consisting of the main original equation and of an adjoint stochastic equations of the same type as the main one. We show how to find $\varepsilon$-optimal strategies and policies in these classes by using the solvability in Sobolev…
In this paper we investigate feedback control techniques for the COVID-19 pandemic which are able to guarantee that the capacity of available intensive care unit beds is not exceeded. The control signal models the social distancing policies enacted by the local government. We propose a control design based on the bang-bang funnel controller which is robust with respect to uncertainties in the parameters of the epidemiological model and only requires measurements of the number of individuals who require medical attention. Therefore, it may serve as a first step towards a reliable decision makin…
Univ. Lille, CNRS, Inria, Centrale Lille, UMR 9189 CRIStAL（里尔大学、法国国家科学研究中心、法国国家信息与自动化研究所、里尔中央理工学院、CRISTAL实验室）；Google Research, Brain Team（谷歌研究院大脑团队）；DeepMind Paris（巴黎深度思维）
Mean Field Games (MFGs) can potentially scale multi-agent systems to extremely large populations of agents. Yet, most of the literature assumes a single initial distribution for the agents, which limits the practical applications of MFGs. Machine Learning has the potential to solve a wider diversity of MFG problems thanks to generalizations capacities. We study how to leverage these generalization properties to learn policies enabling a typical agent to behave optimally against any population distribution. In reference to the Master equation in MFGs, we coin the term ``Master policies'' to des…
One of the most widely used models for studying the geographical economics of climate change is the Regional Integrated model of Climate and the Economy (RICE). In this paper, we investigate how cooperation and competition arise in regional climate policies under the RICE framework from the standpoints of game theory and optimal control. First, we show that the RICE model is inherently a dynamic game. Second, we study both cooperative and non-cooperative solutions to this RICE dynamic game. In cooperative settings, we investigate the global social welfare equilibrium that maximizes the weighte…
Peking University（北京大学）；University of Groningen（格罗宁根大学）
In stochastic dynamic environments, team Markov games have emerged as a versatile paradigm for studying sequential decision-making problems of fully cooperative multi-agent systems. However, the optimality of the derived policies is usually sensitive to model parameters, which are typically unknown and required to be estimated from noisy data in practice. To mitigate the sensitivity of optimal policies to these uncertain parameters, we propose a robust model of team Markov games in this paper, where agents utilize robust optimization approaches to update strategies. This model extends team Mar…
Information Processing and Telecommunications Center（信息处理与电信中心）；Universidad Politécnica de Madrid（马德里理工大学）
Multiagent systems where agents interact among themselves and with a stochastic environment can be formalized as stochastic games. We study a subclass named Markov potential games (MPGs) that appear often in economic and engineering applications when the agents share a common resource. We consider MPGs with continuous state-action variables, coupled constraints and nonconvex rewards. Previous analysis followed a variational approach that is only valid for very simple cases (convex rewards, invertible dynamics, and no coupled constraints); or considered deterministic dynamics and provided open-…
University of Illinois Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）
We consider a subclass of $n$-player stochastic games, in which players have their own internal state/action spaces while they are coupled through their payoff functions. It is assumed that players' internal chains are driven by independent transition probabilities. Moreover, players can receive only realizations of their payoffs, not the actual functions, and cannot observe each other's states/actions. For this class of games, we first show that finding a stationary Nash equilibrium (NE) policy without any assumption on the reward functions is interactable. However, for general reward functio…
Using game and probability theories, I study the French popular game 421, a perfect information stochastic stage game. The problem is to find strategies maximizing the probability of some expected utility. I only solve a player's round against providence, a problem of fate stochastic management: beyond the backward induction solution, bounded complexity motivates heuristic policies. For a unique goal utility, a simple optimal policy, ratchet, is obtained. Its result probabilities are compiled and used, for arbitrary utilities, as the logic of goal identification policies. Various policies appe…
During the Covid-19 pandemic a key role is played by vaccination to combat the virus. There are many possible policies for prioritizing vaccines, and different criteria for optimization: minimize death, time to herd immunity, functioning of the health system. Using an age-structured population compartmental finite-dimensional optimal control model, our results suggest that the eldest to youngest vaccination policy is optimal to minimize deaths. Our model includes the possible infection of vaccinated populations. We apply our model to real-life data from the US Census for New Jersey and Florida…
This paper considers dynamic (multi-stage) signaling games involving an encoder and a decoder who have subjective models on the cost functions. We consider both Nash (simultaneous-move) and Stackelberg (leader-follower) equilibria of dynamic signaling games under quadratic criteria. For the multi-stage scalar cheap talk, we show that the final stage equilibrium is always quantized and under further conditions the equilibria for all time stages must be quantized. In contrast, the Stackelberg equilibria are always fully revealing. In the multi-stage signaling game where the transmission of a Gau…
Successful predictive modeling of epidemics requires an understanding of the implicit feedback control strategies which are implemented by populations to modulate the spread of contagion. While this task of capturing endogenous behavior can be achieved through intricate modeling assumptions, we find that a population's reaction to case counts can be described through a second order affine dynamical system with linear control which fits well to the data across different regions and times throughout the COVID-19 pandemic. The model fits the data well both in and out of sample across the 50 state…
The recent COVID-19 pandemic highlighted the need of non-pharmaceutical interventions in the first stages of a pandemic. Among these, lockdown policies proved unavoidable yet extremely costly from an economic perspective. To better understand the tradeoffs between economic and epidemic costs of lockdown interventions, we here focus on a simple SIR epidemic model and study lockdowns as solutions to an optimal control problem. We first show numerically that the optimal lockdown policy exhibits a phase transition from suppression to mitigation as the time horizon grows, i.e., if the horizon is sh…
The COVID-19 pandemic is the profoundest health crisis of the 21rst century. The SARS-CoV-2 virus arrived in Brazil around March, 2020 and its social and economical backlashes are catastrophic. In this paper, it is investigated how Model Predictive Control (MPC) could be used to plan appropriate social distancing policies to mitigate the pandemic effects in Bahia and Santa Catarina, two states of different regions, culture, and population demography in Brazil. In addition, the parameters of Susceptible-Infected-Recovered-Deceased (SIRD) models for these two states are identified using an optim…
The COVID-19 pandemic has posed a policy making crisis where efforts to slow down or end the pandemic conflict with economic priorities. This paper provides mathematical analysis of optimal disease control policies with idealized compartmental models for disease propagation and simplistic models of social and economic costs. Two locally optimal control strategies are found and categorized as `suppression' and `mitigation' strategies. We analyze how these strategies change when we incorporate vaccination into the model and find a new optimal `delay-mitigation' strategy.
