Open-loop and feedback Nash equilibrium in scalar linear-state differential games with impulse control
Utsav Sadana, Puduru Viswanadha Reddy, Georges Zaccour
备注：
This paper is being withdrawn because some of the results are incorrect. In particular, it can be shown using a reformulation that the control problems of the two players are decoupled
We consider a two-player linear-state differential game, where one player intervenes continuously in the game, while the other implements an impulse control. When the impulse instants are exogenous, we obtain the classical result in linear-state differential games that open-loop and feedback Nash equilibria coincide. When the impulse instants are endogenous, we show that this result no longer holds. In particular, the two equilibria are different. We show that the impulse level is a constant in both equilibria. More importantly, in the open-loop case, we show that the equilibrium number of impulses is at most one, while there can be at most two impulses in the feedback case.
We consider a two-player zero-sum deterministic differential game where each player uses both continuous and impulse controls in infinite-time horizon. We assume that the impulses supposed to be of general term and the costs depend on the state of the system. We use the dynamic programming principle and viscosity solutions approach to show existence and uniqueness of a solution for the Hamilton-Jacobi-Bellman-Isaacs (HJBI) partial differential equations (PDEs) of the game. We prove under Isaacs condition that the upper and lower value functions coincide.
In this paper we consider continuity of the set of Nash equilibria and approximate Nash equilibria for parameterized games. For parameterized games with unique Nash equilibria, the continuity of this equilibrium mapping is well-known. However, when the equilibria need not be unique, there may exist discontinuities in the equilibrium mapping. The focus of this work is to summarize continuity properties for parameterized Nash equilibria and prove continuity via the approximate Nash game with uniformly continuous objective functions over potentially non-compact strategy spaces.
We observe that a significant class of Nash equilibrium problems in non-potential games can be associated with monotone inclusion problems. We propose splitting techniques to solve such problems and establish their convergence. Applications to generalized Nash equilibria, zero-sum games, and cyclic proximity problems are demonstrated.
We investigate the set of Nash equilibrium payoffs for two person differential games. The main result of the paper is the characterization of the set of Nash equilibrium payoffs in the terms of nonsmooth analysis. Also we obtain the sufficient conditions for a pair of continuous function to provide a Nash equilibrium. This result generalizes the method of system of Hamilton-Jacobi equations.
This paper focuses on zero-sum stochastic differential games in the framework of forward-backward stochastic differential equations on a finite time horizon with both players adopting impulse controls. By means of BSDE methods, in particular that of the notion from Peng's stochastic \textit{% backward semigroups}, we prove a dynamic programming principle for both the upper and the lower value functions of the game. The upper and the lower value functions are then shown to be the unique viscosity solutions of the Hamilton-Jacobi-Bellman-Isaacs equations with a double-obstacle. As a consequence,…
We consider a class of N-player stochastic games of multi-dimensional singular control, in which each player faces a minimization problem of monotone-follower type with submodular costs. We call these games "monotone-follower games". In a not necessarily Markovian setting, we establish the existence of Nash equilibria. Moreover, we introduce a sequence of approximating games by restricting, for each natural number n, the players' admissible strategies to the set of Lipschitz processes with Lipschitz constant bounded by n. We prove that, for each n, there exists a Nash equilibrium of the approx…
We present a unified framework for characterizing local Nash equilibria in continuous games on either infinite-dimensional or finite-dimensional non-convex strategy spaces. We provide intrinsic necessary and sufficient first- and second-order conditions ensuring strategies constitute local Nash equilibria. We term points satisfying the sufficient conditions differential Nash equilibria. Further, we provide a sufficient condition (non-degeneracy) guaranteeing differential Nash equilibria are isolated and show that such equilibria are structurally stable. We present tutorial examples to illustra…
Distributed Nash equilibrium seeking of aggregative games is investigated and a continuous-time algorithm is proposed. The algorithm is designed by virtue of projected gradient play dynamics and distributed average tracking dynamics, and is applicable to games with constrained strategy sets and weight-balanced communication graphs. We obtain an exponential convergence of the proposed algorithm to the Nash equilibrium. Numerical examples illustrate the effectiveness of our methods.
In this paper we introduce the concept of split Nash equilibrium problems associated with two related noncooperative strategic games. Then we apply the Fan-KKM theorem to prove the existence of solutions to split Nash equilibrium problems of related noncooperative strategic games, in which the strategy sets of the players are nonempty closed and convex subsets in Banach spaces. As application of this existence to economics, an example is provided to study the existence of split Nash equilibrium of utilities of two related economies. As applications, we study the existence of split Nash equilib…
In the present paper, we study a two-player zero-sum deterministic differential game with both players adopting impulse controls, in infinite time horizon, under rather weak assumptions on the cost functions. We prove by means of the dynamic programming principle (DPP) that the lower and upper value functions are continuous and viscosity solutions to the corresponding Hamilton-Jacobi-Bellman-Isaacs (HJBI) quasi-variational inequality (QVI). We define a new HJBI QVI for which, under a proportional property assumption on the maximizer cost, the value functions are the unique viscosity solution. …
This paper considers a new class of deterministic finite-time horizon, two-player, zero-sum differential games (DGs) in which the maximizing player is allowed to take continuous and impulse controls whereas the minimizing player is allowed to take impulse control only. We seek to approximate the value function, and to provide a verification theorem for this class of DGs. We first, by means of dynamic programming principle (DPP) in viscosity solution (VS) framework, characterize the value function as the unique VS to the related Hamilton-Jacobi-Bellman-Isaacs (HJBI) double-obstacle equation. Ne…
For a mean field game model with a major and infinite minor players, we characterize a notion of Nash equilibrium via a system of so-called master equations, namely a system of nonlinear transport equations in the space of measures. Then, for games with a finite number N of minor players and a major player, we prove that the solution of the corresponding Nash system converges to the solution of the system of master equations as N tends to infinity.
In this note we are interested in a relevant generalized Nash equilibrium problem, which was proposed by Rosen in 1965. An existence result is established in the general setting of quasiconvexity, which is independent from the one given by Aussel and Dutta in 2008.
We consider a stochastic differential game in the context of forward-backward stochastic differential equations, where one player implements an impulse control while the opponent controls the system continuously. Utilizing the notion of "backward semigroups" we first prove the dynamic programming principle (DPP) for a truncated version of the problem in a straightforward manner. Relying on a uniform convergence argument then enables us to show the DPP for the general setting. In particular, this avoids technical constraints imposed in previous works dealing with the same problem. Moreover, our…
In this paper, we study a nonzero-sum stochastic differential game in Markovian framework. We show the existence of the Nash equilibrium point which is discontinuous and of bang-bang type under natural conditions. The main tool is the notion of backward stochastic differential equations which, in our case, are multidimensional with discontinuous generators with respect to z component.
We consider multi-agent decision making, where each agent optimizes its cost function subject to constraints. Agents' actions belong to a compact convex Euclidean space and the agents' cost functions are coupled. We propose a distributed payoff-based algorithm to learn Nash equilibria in the game between agents. Each agent uses only information about its current cost value to compute its next action. We prove convergence of the proposed algorithm to a Nash equilibrium in the game leveraging established results on stochastic processes. The performance of the algorithm is analyzed with a numeric…
We establish the existence of fixed points for set-valued maps defined on metric spaces and satisfying a pointwise or a local version of Banach's contraction property. As an application, we demonstrate the existence of Nash equilibrium in a general class of strategic games played on metric spaces of strategies.
We consider multi-agent decision making where each agent optimizes its convex cost function subject to individual and coupling constraints. The constraint sets are compact convex subsets of a Euclidean space. To learn Nash equilibria, we propose a novel distributed payoff-based algorithm, such that each agent uses information only about its cost function values and the constraint function values with their associated dual multiplier. We prove convergence of this algorithm to a Nash equilibrium, under the assumption that the game admits a strictly convex potential function. In the absence of co…
We consider an investment problem in which an investor performs capital injections to increase the liquidity of a firm for it to maximise profit from market operations. Each time the investor performs an injection, the investor incurs a fixed transaction cost. In addition to maximising their terminal reward, the investor seeks to minimise the risk of loss of their investment (from a possible firm ruin) by exiting the market at some point in time. We show that the problem can be reformulated in terms of a new stochastic differential game of control and stopping in which one of the players modif…
This paper studies an asymptotic solvability problem for linear quadratic (LQ) mean field games with controlled diffusions and indefinite weights for the state and control in the costs. We employ a rescaling approach to derive a low dimensional Riccati ordinary differential equation (ODE) system, which characterizes a necessary and sufficient condition for asymptotic solvability. The rescaling technique is further used for performance estimates, establishing an $O(1/N)$-Nash equilibrium for the obtained decentralized strategies.
In this paper, we aim to design a distributed approximate algorithm for seeking Nash equilibria of an aggregative game. Due to the local set constraints of each player, projectionbased algorithms have been widely employed for solving such problems actually. Since it may be quite hard to get the exact projection in practice, we utilize inscribed polyhedrons to approximate local set constraints, which yields a related approximate game model. We first prove that the Nash equilibrium of the approximate game is the $ε$-Nash equilibrium of the original game, and then propose a distributed algorithm …
We design the first fully-distributed algorithm for generalized Nash equilibrium seeking in aggregative games on a time-varying communication network, under partial-decision information, i.e., the agents have no direct access to the aggregate decision. The algorithm is derived by integrating dynamic tracking into a projected pseudo-gradient algorithm. The convergence analysis relies on the framework of monotone operator splitting and the Krasnosel'skii-Mann fixed-point iteration with errors.
We consider a controlled linear-quadratic (LQ) large-population system with mixture of three types agents: major leader, minor leaders and minor followers. The Stackelberg-Nash-Cournot (SNC) approximate equilibrium is studied by a major-minor mean-field game (MFG) coupled with a leader-follower Stackelberg game. By variational method, the SNC approximate equilibrium strategy can be represented by some forward-backward-stochastic-differential-equations (FBSDEs) in the open-loop sense. And we pay great effort to give the feedback form of the open-loop strategy by some Riccati equations.
The theory of mean field games is a tool to understand noncooperative dynamic stochastic games with a large number of players. Much of the theory has evolved under conditions ensuring uniqueness of the mean field game Nash equilibrium. However, in some situations, typically involving symmetry breaking, non-uniqueness of solutions is an essential feature. To investigate the nature of non-unique solutions, this paper focuses on the technically simple setting where players have one of two states, with continuous time dynamics, and the game is symmetric in the players, and players are restricted t…
We consider a general class of nonzero-sum $N$-player stochastic games with impulse controls, where players control the underlying dynamics with discrete interventions. We adopt a verification approach and provide sufficient conditions for the Nash equilibria (NEs) of the game. We then consider the limit situation of $N \to \infty$, that is, a suitable mean-field game (MFG) with impulse controls. We show that under appropriate technical conditions, the existence of unique NE solution to the MFG, which is an $ε$-NE approximation to the $N$-player game, with $ε=O\left(\frac{1}{\sqrt{N}}\right)$.…
In the nonzero-sum setting, we establish a connection between Nash equilibria in games of optimal stopping (Dynkin games) and generalised Nash equilibrium problems (GNEP). In the Dynkin game this reveals novel equilibria of threshold type and of more complex types, and leads to novel uniqueness and stability results.
A bimatrix game $(A,B)$ is called a game of rank $k$ if the rank of the matrix $A+B$ is at most $k$. We consider the problem of enumerating the Nash equilibria in (non-degenerate) games of rank 1. In particular, we show that even for games of rank 1 not all equilibria can be reached by a Lemke-Howson path and present a parametric simplex-type algorithm for enumerating all Nash equilibria of a non-degenerate game of rank 1.
It is known that the generalized Nash equilibrium problem can be reformulated as a quasivariational inequality. Our aim in this work is to introduce a variational approach to study the existence of solutions for generalized ordinal Nash games, that is, generalized games where the player preferences are binary relations that do not necessarily admit a utility representation.
This paper considers the problem of two-player zero-sum stochastic differential game with both players adopting impulse controls in finite horizon under rather weak assumptions on the cost functions ($c$ and $χ$ not decreasing in time). We use the dynamic programming principle and viscosity solutions approach to show existence and uniqueness of a solution for the Hamilton-Jacobi-Bellman-Isaacs (HJBI) partial differential equation (PDE) of the game. We prove that the upper and lower value functions coincide.
If a game has a unique Nash equilibrium, then this equilibrium is arguably the solution of the game from the refinement's literature point of view. However, it might be that for almost all initial conditions, all strategies in the support of this equilibrium are eliminated by the replicator dynamics and the best-reply dynamics.
In this paper, we investigate Nash equilibrium payoffs for nonzero-sum stochastic differential games with reflection. We obtain an existence theorem and a characterization theorem of Nash equilibrium payoffs for nonzero-sum stochastic differential games with nonlinear cost functionals defined by doubly controlled reflected backward stochastic differential equations.
Discretely-constrained Nash-Cournot games have attracted attention as they arise in various competitive energy production settings in which players must make one or more discrete decisions. Gabriel et al. ["Solving discretely-constrained Nash-Cournot games with an application to power markets." Networks and Spatial Economics 13(3), 2013] claim that the set of equilibria to a discretely-constrained Nash-Cournot game coincides with the set of solutions to a corresponding discretely-constrained mixed complementarity problem. We show that this claim is false.
This paper develops a novel methodology to study robust stability properties of Nash equilibrium points in dynamic games. Small-gain techniques in modern mathematical control theory are used for the first time to derive conditions guaranteeing uniqueness and global asymptotic stability of Nash equilibrium point for economic models described by functional difference equations. Specification to a Cournot oligopoly game is studied in detail to demonstrate the power of the proposed methodology.
In this article, we consider generalized Nash games where the associated constraint map is not necessarily self. The classical Nash equilibrium may not exist for such games and therefore we introduce the notion of best approximate solution for such games. We investigate the occurrence of best approximate solutions for such generalized Nash games consisting of infinitely many players in which each player regulates the strategy variable lying in a topological vector space. Based on the maximum theorem and a fixed point result for Kakutani factorizable maps, we derive the existence of best approx…
We study a class of dynamic decision problems of mean field type with time inconsistent cost functionals, and derive a stochastic maximum principle to characterize subgame perfect Nash equilibrium points. Subsequently, this approach is extended to a mean field game to construct decentralized strategies and obtain an estimate of their performance.
This paper is related to nonzero-sum stochastic differential games in the Markovian framework. We show existence of a Nash equilibrium point for the game when the drift is no longer bounded and only satisfies a linear growth condition. The main tool is the notion of backward stochastic differential equations which, in our case, are multidimensional with continuous coefficient and stochastic linear growth.
We study a new kind of non-zero-sum stochastic differential game with mixed impulse/switching controls, motivated by strategic competition in commodity markets. A representative upstream firm produces a commodity that is used by a representative downstream firm to produce a final consumption good. Both firms can influence the price of the commodity. By shutting down or increasing generation capacities, the upstream firm influences the price with impulses. By switching (or not) to a substitute, the downstream firm influences the drift of the commodity price process. We study the resulting impul…
In this paper, we study Nash equilibrium payoffs for nonzero-sum stochastic differential games via the theory of backward stochastic differential equations. We obtain an existence theorem and a characterization theorem of Nash equilibrium payoffs for nonzero-sum stochastic differential games with nonlinear cost functionals defined with the help of a doubly controlled backward stochastic differential equation. Our results extend former ones by Buckdahn, Cardaliaguet and Rainer (2004) and are based on a backward stochastic differential equation approach.
We study a two-player nonzero-sum stochastic differential game where one player controls the state variable via additive impulses while the other player can stop the game at any time. The main goal of this work is characterize Nash equilibria through a verification theorem, which identifies a new system of quasi-variational inequalities whose solution gives equilibrium payoffs with the correspondent strategies. Moreover, we apply the verification theorem to a game with a one-dimensional state variable, evolving as a scaled Brownian motion, and with linear payoff and costs for both players. Two…
We study stochastic mean-field games among finite number of teams with large finite as well as infinite number of decision makers. For this class of games within static and dynamic settings, we establish the existence of a Nash equilibrium, and show that a Nash equilibrium exhibits exchangeability in the finite decision maker regime and symmetry in the infinite one. To arrive at these existence and structural theorems, we endow the set of randomized policies with a suitable topology under various decentralized information structures, which leads to the desired convexity and compactness of the …
This paper aims to design a distributed coordination algorithm for solving a multi-agent decision problem with a hierarchical structure. The primary goal is to search the Nash equilibrium of a noncooperative game such that each player has no incentive to deviate from the equilibrium under its private objective. Meanwhile, the agents can coordinate to optimize the social cost within the set of Nash equilibria of the underlying game. Such an optimal Nash equilibrium problem can be modeled as a distributed optimization problem with variational inequality constraints. We consider the scenario wher…
In this paper, we introduce the concept of infinitely split Nash equilibrium in repeated games in which the profile sets are chain-complete posets. Then by using a fixed point theorem on posets in [8], we prove an existence theorem. As an application, we study the repeated extended Bertrant duopoly model of price competition.
We consider both discrete and continuous time finite state-action stochastic games. In discrete time stochastic games, it is known that a stationary Blackwell-Nash equilibrium (BNE) exists for a single controller additive reward (SC-AR) stochastic game which is a special case of a general stochastic game. We show that, in general, the additive reward condition is needed for the existence of a BNE. We give an example of a single controller stochastic game which does not satisfy additive reward condition. We show that this example does not have a stationary BNE. For a general discrete time disco…
University of California, Berkeley（加州大学伯克利分校）；Department of Electrical Engineering and Computer Science（电气工程与计算机科学系）；Division of Computer Science（计算机科学部）；Department of Statistics（统计学系）
We propose local symplectic surgery, a two-timescale procedure for finding local Nash equilibria in two-player zero-sum games. We first show that previous gradient-based algorithms cannot guarantee convergence to local Nash equilibria due to the existence of non-Nash stationary points. By taking advantage of the differential structure of the game, we construct an algorithm for which the local Nash equilibria are the only attracting fixed points. We also show that the algorithm exhibits no oscillatory behaviors in neighborhoods of equilibria and show that it has the same per-iteration complexit…
