Static potential games are non-cooperative games which admit a fictitious function, also referred to as a potential function, such that the minimizers of this function constitute a subset (or a refinement) of the Nash equilibrium strategies of the associated non-cooperative game. In this paper, we study a class $N$-player non-zero sum difference games with inequality constraints which admit a potential game structure. In particular, we provide conditions for the existence of an optimal control problem (with inequality constraints) such that the solution of this problem yields an open-loop Nash equilibrium strategy of the corresponding dynamic non-cooperative game (with inequality constraints). Further, we provide a way to construct potential functions associated with this optimal control problem. We specialize our general results to a linear-quadratic setting and provide a linear complementarity problem-based approach for computing the refinements of the open-loop Nash equilibria. We illustrate our results with an example inspired by energy storage incentives in a smart grid.
In this work, we study the distributed Nash equilibrium seeking problem for monotone generalized noncooperative games with set constraints and shared affine inequality constraints. A distributed regularized penalty method is proposed. The idea is to use a differentiable penalty function with a time-varying penalty parameter to deal with the inequality constraints. A time-varying regularization term is used to deal with the ill-poseness caused by the monotonicity assumption and the time-varying penalty term. The asymptotic convergence to the least-norm variational equilibrium of the game is pro…
In this paper, the known deterministic linear-quadratic Stackelberg game is revisited, whose open-loop Stackelberg solution actually possesses the nature of time inconsistency. To handle this time inconsistency, {a two-tier game framework is introduced, where the upper-tier game works according to Stackelberg's scenario with a leader and a follower, and two lower-tier intertemporal games give the follower's and leader's equilibrium response mappings that mimic the notion of time-consistent open-loop equilibrium control in existing literature. The resulting open-loop equilibrium solution of the…
In this paper, we consider a linear quadratic stochastic two-person nonzero-sum differential game. Open-loop and closed-loop Nash equilibria are introduced. The existence of the former is characterized by the solvability of a system of forward-backward stochastic differential equations, and that of the latter is characterized by the solvability of a system of coupled symmetric Riccati differential equations. Sometimes, open-loop Nash equilibria admit a closed-loop representation, via the solution to a system of non-symmetric Riccati equations, which is different from the outcome of the closed-…
In this paper, the open-loop and closed-loop local and remote stochastic nonzero-sum game (LRSNG) problem is investigated. Different from previous works, the stochastic nonzero-sum game problem under consideration is essentially a special class of two-person nonzero-sum game problem, in which the information sets accessed by the two players are inconsistent. More specifically, both the local player and the remote player are involved in the system dynamics, and the information sets obtained by the two players are different, and each player is designed to minimize its own cost function. For the …
The paper is concerned with two-person zero-sum mean-field linear-quadratic stochastic differential games over finite horizons. By a Hilbert space method, a necessary condition and a sufficient condition are derived for the existence of an open-loop saddle point. It is shown that under the sufficient condition, the associated two Riccati equations admit unique strongly regular solutions, in terms of which the open-loop saddle point can be represented as a linear feedback of the current state. When the game only satisfies the necessary condition, an approximate sequence is constructed by solvin…
In this paper, we consider a linear quadratic stochastic two-person zero-sum differential game. The controls for both players are allowed to appear in both drift and diffusion of the state equation. The weighting matrices in the performance functional are not assumed to be definite/non-singular. A necessary and sufficient condition for the existence of a closed-loop saddle point is established in terms of the solvability of a Riccati differential equation with certain regularity. It is possible that the closed-loop saddle point fails to exist, and at the same time, the corresponding Riccati eq…
This paper is concerned with the closed-loop solvability of one kind of linear-quadratic Stackelberg stochastic differential game, where the coefficients are deterministic. The notion of the closed-loop solvability is introduced, which require to be independent of the initial state. The follower's problem is solved first, and the closed-loop optimal strategy is characterized by a Riccati equation, together with an adapted solution to a linear backward stochastic differential equation. Then the necessary conditions of the existence of the leader's nonanticipating closed-loop optimal strategy is…
A defender-attacker-target problem with non-moving target is considered. This problem is modeled by a pursuit-evasion zero-sum differential game with linear dynamics and quadratic cost functional. In this game the pursuer is the defender, while the evader is the attacker. The objective of the pursuer is to minimize the cost functional, while the evader has two objectives: to maximize the cost functional and to keep a given terminal state inequality constraint. The open-loop saddle point solution of this game is obtained in the case where the transfer functions of the controllers for the defend…
We consider a two-player linear-state differential game, where one player intervenes continuously in the game, while the other implements an impulse control. When the impulse instants are exogenous, we obtain the classical result in linear-state differential games that open-loop and feedback Nash equilibria coincide. When the impulse instants are endogenous, we show that this result no longer holds. In particular, the two equilibria are different. We show that the impulse level is a constant in both equilibria. More importantly, in the open-loop case, we show that the equilibrium number of imp…
This paper obtains the maximum principle for both stochastic (global) open-loop and stochastic (global) closed-loop Stackelberg differential games. For the closed-loop case, we use the theory of controlled forward-backward stochastic differential equations to derive the maximum principle for the leader's optimal strategy. In the special case of the open-loop linear quadratic Stackelberg game, we consider the follower's Hamiltonian system as the leader's state equation, derive the related stochastic Riccati equation, and show the existence and uniqueness of the solution to the Riccati equation …
The paper studies the open-loop saddle point and the open-loop lower and upper values, as well as their relationship for two-person zero-sum stochastic linear-quadratic (LQ, for short) differential games with deterministic coefficients. It derives a necessary condition for the finiteness of the open-loop lower and upper values and a sufficient condition for the existence of an open-loop saddle point. It turns out that under the sufficient condition, a strongly regular solution to the associated Riccati equation uniquely exists, in terms of which a closed-loop representation is further establis…
This paper is concerned with two-person mean-field linear-quadratic non-zero sum stochastic differential games in an infinite horizon. Both open-loop and closed-loop Nash equilibria are introduced. Existence of an open-loop Nash equilibrium is characterized by the solvability of a system of mean-field forward-backward stochastic differential equations in an infinite horizon and the convexity of the cost functionals, and the closed-loop representation of an open-loop Nash equilibrium is given through the solution to a system of two coupled non-symmetric algebraic Riccati equations. The existenc…
In this paper, we present a game-theoretic model describing voluntary social distancing during the spread of an epidemic. The payoffs of the agents depend on the social distancing they practice and on the probability of getting infected. We consider two types of agents, the non-vulnerable agents who have a small cost if they get infected, and the vulnerable agents who have a higher cost. For the modeling of the epidemic outbreak, we consider a variant of the SIR (Susceptible-Infected-Removed) model involving populations of susceptible, infected, and removed persons of vulnerable and non-vulner…
This paper is concerned with a linear quadratic stochastic two-person zero-sum differential game with constant coefficients in an infinite time horizon. Open-loop and closed-loop saddle points are introduced. The existence of closed-loop saddle points is characterized by the solvability of an algebraic Riccati equation with a certain stabilizing condition. A crucial result makes our approach work is the unique solvability of a class of linear backward stochastic differential equations in an infinite horizon.
This paper aims to provide directly the observability inequality of backward stochastic heat equations for measurable sets. As an immediate application, the null controllability of the forward heat equations is obtained. Moreover, an interesting relaxed optimal actuator location problem is formulated, and the existence of its solution is proved. Finally, the solution is characterized by a Nash equilibrium of the associated game problem.
In this paper, a leader-follower stochastic differential game is studied for a linear stochastic differential equation with a quadratic cost functional. The coefficients in the state equation and the weighting matrices in the cost functionals are all deterministic. Closed-loop strategies are introduced, which require to be independent of initial states; and such a nature makes it very useful and convenient in applications. The follower first solves a stochastic linear quadratic optimal control problem, and his optimal closed-loop strategy is characterized by a Riccati equation, together with a…
A conditional-gradient-based single-loop augmented Lagrangian method for inequality constrained optimization
School of Mathematical Sciences, South China Normal University（华南师范大学数学科学学院）；Department of Applied Mathematics, The Hong Kong Polytechnic University（香港理工大学应用数学系）；Department of Mathematics and Statistics, James Madison University（詹姆斯麦迪逊大学数学与统计系）
Variational inequality problems allow for capturing an expansive class of problems, including convex optimization problems, convex Nash games and economic equilibrium problems, amongst others. Yet in most practical settings, such problems are complicated by uncertainty, motivating the examination of a stochastic generalization of the variational inequality problem and its extensions in which the components of the mapping contain expectations. When the associated sets are unbounded, ascertaining existence requires having access to analytical forms of the expectations. Naturally, in practical se…
We consider a controlled linear-quadratic (LQ) large-population system with mixture of three types agents: major leader, minor leaders and minor followers. The Stackelberg-Nash-Cournot (SNC) approximate equilibrium is studied by a major-minor mean-field game (MFG) coupled with a leader-follower Stackelberg game. By variational method, the SNC approximate equilibrium strategy can be represented by some forward-backward-stochastic-differential-equations (FBSDEs) in the open-loop sense. And we pay great effort to give the feedback form of the open-loop strategy by some Riccati equations.
This paper analyses a stochastic differential game of control and stopping in which one of the players modifies a diffusion process using impulse controls, an adversary then chooses a stopping time to end the game. The paper firstly establishes the regularity and boundedness of the upper and lower value functions from which an appropriate variant of the dynamic programming principle (DPP) is derived. It is then proven that the upper and lower value functions coincide so that the game admits a value and that the value of the game is a unique viscosity solution to a HJBI equation described by a …
We consider an extension of a noncooperative game problem where players have joint binding constraints. In this case, justification of a generalized equilibrium point needs a reasonable mechanism for attaining this state. We suggest to combine a penalty method together with shares allocation of right-hand sides, which replaces the initial problem with a sequence of the usual Nash equilibrium problems together with an upper level variational inequality as a master problem. We show convergence of solutions of these auxiliary penalized problems to a solution of the initial game problem under weak…
This paper studies the convergence problem for mean field games with common noise. We define a suitable notion of weak mean field equilibria, which we prove captures all subsequential limit points, as $n\to\infty$, of closed-loop approximate equilibria from the corresponding $n$-player games. This extends to the common noise setting a recent result of the first author, while also simplifying a key step in the proof and allowing unbounded coefficients and non-i.i.d. initial conditions. Conversely, we show that every weak mean field equilibrium arises as the limit of some sequence of approximate…
In this article, we concern a kind of partially observed non-zero sum stochastic differential game based on forward and backward stochastic differential equations (FBSDEs). It is required that each player has his own observation equation, and the corresponding open-loop Nash equilibrium control is required to adapted to the filtration that the observation process generated. To find this open-loop Nash equilibrium point, we prove the maximum principle as a necessary condition of the existence of this point, and give a verification theorem as a sufficient condition to verify it is the real open-…
This paper continues the study of the mean field game (MFG) convergence problem: In what sense do the Nash equilibria of $n$-player stochastic differential games converge to the mean field game as $n\rightarrow\infty$? Previous work on this problem took two forms. First, when the $n$-player equilibria are open-loop, compactness arguments permit a characterization of all limit points of $n$-player equilibria as weak MFG equilibria, which contain additional randomness compared to the standard (strong) equilibrium concept. On the other hand, when the $n$-player equilibria are closed-loop, the con…
It is known that the generalized Nash equilibrium problem can be reformulated as a quasivariational inequality. Our aim in this work is to introduce a variational approach to study the existence of solutions for generalized ordinal Nash games, that is, generalized games where the player preferences are binary relations that do not necessarily admit a utility representation.
Large population games with interactions through controls and common noise: convergence results and equivalence between $open$--$loop$ and $closed$--$loop$ controls
Dynkin Game of Stochastic Differential Equations with Random Coefficients, and Associated Backward Stochastic Partial Differential Variational Inequality
South China Normal University（华南师范大学）；Fudan University（复旦大学）；Ajou University（阿尤大学）
We present an analysis of advertising activities in a dynamic oligopoly with differentiated goods by differential game approach under general demand and cost functions. Mainly we show the following results. The comparison of the open-loop solution and that of the closed-loop solution depends on whether the outputs of the firms are strategic substitutes or strategic complements, and the memoryless closed-loop solution and the feedback solution are equivalent when there is no spillover effect of advertising activities.
Designing efficient algorithms to compute Nash equilibria poses considerable challenges in Algorithmic Game Theory and Optimization. In this work, we employ integer programming techniques to compute Nash equilibria in Integer Programming Games, a class of simultaneous and non-cooperative games where each player solves a parametrized integer program. We introduce ZERO Regrets, a general and efficient cutting plane algorithm to compute, enumerate, and select Nash equilibria. Our framework leverages the concept of equilibrium inequality, an inequality valid for any Nash equilibrium, and the assoc…
Information Processing and Telecommunications Center（信息处理与电信中心）；Universidad Politécnica de Madrid（马德里理工大学）
Multiagent systems where agents interact among themselves and with a stochastic environment can be formalized as stochastic games. We study a subclass named Markov potential games (MPGs) that appear often in economic and engineering applications when the agents share a common resource. We consider MPGs with continuous state-action variables, coupled constraints and nonconvex rewards. Previous analysis followed a variational approach that is only valid for very simple cases (convex rewards, invertible dynamics, and no coupled constraints); or considered deterministic dynamics and provided open-…
