In this paper, we consider the stochastic optimal control problem for the interacting particle system. We obtain the stochastic maximum principle of the optimal control system by introducing a generalized backward stochastic differential equation with interaction. The existence and uniqueness of the solution of this type of equation is proved. We derive the necessary condition that the optimal control should satisfy. As an application, the linear quadratic case is investigated to illustrate the main results.
This work exhibits a novel phase transition for the classical stochastic block model (SBM). In addition we study the SBM in the corresponding near-critical regime, and find the scaling limit for the component sizes. The two-parameter stochastic process arising in the scaling limit, an analogue of the standard Aldous' multiplicative coalescent, is interesting in its own right. We name it the (standard) Interacting Multiplicative Coalescent. To the best of our knowledge, this object has not yet appeared in the literature.
In this article we study the convergence of a stochastic particle system that interacts through threshold hitting times towards a novel equation of McKean-Vlasov type. The particle system is motivated by an original model for the behavior of a network of neurons, in which a classical noisy integrate-and-fire model is coupled with a cable equation to describe the dendritic structure of each neuron.
For a system of weakly interacting anharmonic oscillators, perturbed by an energy preserving stochastic dynamics, we prove an autonomous (stochastic) evolution for the energies at large time scale (with respect to the coupling parameter). It turn out that this macroscopic evolution is given by the so called conservative (non-gradient) Ginzburg-Landau system of stochastic differential equations. The proof exploits hypocoercivity and hypoellipticity properties of the uncoupled dynamics.
Consider $(X_{i}(t))$ solving a system of $N$ stochastic differential equations interacting through a random matrix $\mathbf J = (J_{ij})$ with independent (not necessarily identically distributed) random coefficients. We show that the trajectories of averaged observables of $(X_i(t))$, initialized from some $μ$ independent of $\mathbf J$, are universal, i.e., only depend on the choice of the distribution $\mathbf{J}$ through its first and second moments (assuming e.g., sub-exponential tails). We take a general combinatorial approach to proving universality for dynamical systems with random co…
In this paper we are concerned with the maximum principle for quasi-linear backward stochastic partial differential equations (BSPDEs for short) of parabolic type. We first prove the existence and uniqueness of the weak solution to quasi-linear BSPDE with the null Dirichlet condition on the lateral boundary. Then using the De Giorgi iteration scheme, we establish the maximum estimates and the global maximum principle for quasi-linear BSPDEs. To study the local regularity of weak solutions, we also prove a local maximum principle for the backward stochastic parabolic De Giorgi class.
We prove a maximum principle for local solutions of quasilinear stochastic PDEs with obstacle (in short OSPDE). The proofs are based on a version of Itô's formula and estimates for the positive part of a local solution which is non-positive on the lateral boundary.
We introduce and study a class of infinite-horizon non-zero-sum non-cooperative stochastic games with infinitely many interacting agents using ideas of statistical mechanics. First we show, in the general case of asymmetric interactions, the existence of a strategy that allows any player to eliminate losses after a finite random time. In the special case of symmetric interactions, we also prove that, as time goes to infinity, the game converges to a Nash equilibrium. Moreover, assuming that all agents adopt the same strategy, using arguments related to those leading to perfect simulation algor…
In this paper, we consider particle systems with interaction and Brownian motion. We prove that when the initial data is from the sampling of Chorin's method, i.e., the initial vertices are on lattice points $hi\in \mathbb{R}^d$ with mass $ρ_0(hi) h^d$, where $ρ_0$ is some initial density function, then the regularized empirical measure of the interacting particle system converges in probability to the corresponding mean-field partial differential equation with initial density $ρ_0$, under the Sobolev norm of $L^\infty(L^2)\cap L^2(H^1)$. Our result is true for all those systems when the inter…
We derive sufficient and necessary optimality conditions in terms of a stochastic maximum principle (SMP) for controls associated with cost functionals of mean-field type, under dynamics driven by a class of Markov chains of mean-field type which are pure jump processes obtained as solutions of a well-posed martingale problem. As an illustration, we apply the result to generic examples of control problems as well as some applications.
This paper is concerned with the partial information optimal control problem of wa controlled forward-backward stochastic differential equation of jump diffusion with correlated noises between the system and the observation. For this type of partial information optimal control problem, Necessary and sufficient optimality conditions, in the form of Pontryagin maximum principle, for the partial information optimal control are established using a unified way. Moreover, our admissible control process $u(\cdot)$ satisfies the following integrable condition condition: \begin{equation*} \label{eq:1.1…
This paper concerns the large deviations of a system of interacting particles on a random graph. There is no stochasticity, and the only sources of disorder are the random graph connections, and the initial condition. The average number of afferent edges on any particular vertex must diverge to infinity as $N\to \infty$, but can do so at an arbitrarily slow rate. These results are thus accurate for both sparse and dense random graphs. A particular application to sparse Erdos-Renyi graphs is provided. The theorem is proved by pushing forward a Large Deviation Principle for a `nested empirical m…
We consider a system of stochastic interacting particles in $\mathbb{R}^d$ and we describe large deviations asymptotics in a joint mean-field and small-noise limit. Precisely, a large deviations principle (LDP) is established for the empirical measure and the stochastic current, as the number of particles tends to infinity and the noise vanishes, simultaneously. We give a direct proof of the LDP using tilting and subsequently exploiting the link between entropy and large deviations. To this aim we employ consistency of suitable deterministic control problems associated to the stochastic dynami…
On the long time behaviour of single stochastic Hodgkin-Huxley neurons with constant signal, and a construction of circuits of interacting neurons showing self-organized rhythmic oscillations
We consider stochastic reaction networks modeled by continuous-time Markov chains. Such reaction networks often contain many reactions, potentially occurring at different time scales, and have unknown parameters (kinetic rates, total amounts). This makes their analysis complex. We examine stochastic reaction networks with non-interacting species that often appear in examples of interest (e.g. in the two-substrate Michaelis Menten mechanism). Non-interacting species typically appear as intermediate (or transient) chemical complexes that are depleted at a fast rate. We embed the Markov process o…
We consider an interacting particle system modeled as a system of $N$ stochastic differential equations driven by Brownian motions. We prove that the (mollified) empirical process converges, uniformly in time and space variables, to the solution of the two-dimensional Navier-Stokes equation written in vorticity form. The proofs follow a semigroup approach.
We consider a stochastic interacting particle system in a bounded domain with reflecting boundary, including creation of new particles on the boundary prescribed by a given source term. We show that such particle system approximates 2d Navier-Stokes equations in vorticity form and impermeable boundary, the creation of particles modeling vorticity creation at the boundary. Kernel smoothing, more specifically smoothing by means of the Neumann heat semigroup on the space domain, allows to establish uniform convergence of regularized empirical measures to (weak solutions of) Navier-Stokes equation…
A necessary maximum principle is proved for optimal controls of stochastic systems driven by multidimensional Teugel's martingales. The multidimensional Teugel's martingales are constructed by orthogonalizing the multidimensional Lévy processes. The control domain need not be convex, and the control is allowed to enter into the terms of Teugel's martingales.
We consider a stochastic control problem, where the control domain is convex and the system is governed by a nonlinear backward stochastic differential equation. With a L1 terminal data, we derive necessary optimality conditions in the form of stochastic maximum principle.
We obtain the variational equations for backward stochastic differential equations in recursive stochastic optimal control problems, and then get the maximum principle which is novel. The control domain need not be convex, and the generator of the backward stochastic differential equation can contain z.
We study a stochastic optimal control problem for forward-backward control systems with quadratic generators. In order to establish the first and second-order variational and adjoint equations, we obtain a new estimate for one-dimensional linear BSDEs with unbounded stochastic Lipschitz coefficients involving bounded mean oscillation martingales (BMO-martingales for short) and prove the solvability for a class of multi-dimensional BSDEs with this type. Finally, a new global stochastic maximum principle is deduced.
We consider the stochastic optimal control problem for the dynamical system of the stochastic differential equation driven by a local martingale with a spatial parameter. Assuming the convexity of the control domain, we obtain the stochastic maximum principle as the necessary condition for an optimal control, and we also prove its sufficiency under proper conditions. The stochastic linear quadratic problem in this setting is also discussed.
In this paper, we consider a stochastic recursive optimal control problem under model uncertainty. In this framework, the cost function is described by solutions of a family of backward stochastic differential equations. With the help of the linearization techniques and weak convergence methods, we derive the corresponding stochastic maximum principle. Moreover, a linear quadratic robust control problem is also studied.
We prove a stochastic maximum principle ofPontryagin's type for the optimal control of a stochastic partial differential equationdriven by white noise in the case when the set of control actions is convex. Particular attention is paid to well-posedness of the adjoint backward stochastic differential equation and the regularity properties of its solution with values in infinite-dimensional spaces.
In this paper, we study the maximum principle for stochastic optimal control problems of forward-backward stochastic difference systems (FBSΔSs) where the uncertainty is modeled by a discrete time, finite state process, rather than white noises. Two types of FBSΔSs are investigated. The first one is described by a partially coupled forward-backward stochastic difference equation (FBSΔE) and the second one is described by a fully coupled FBSΔE. By adopting an appropriate representation of the product rule and an appropriate formulation of the backward stochastic difference equation (BSΔE), we d…
Motivated by a problem of optimal harvesting of natural resources, we study a control problem for Volterra type dynamics driven by time-changed Lévy noises, which are in general not Markovian. To exploit the nature of the noise, we make use of different kind of information flows within a maximum principle approach. For this we work with backward stochastic differential equations (BSDE) with time-change and exploit the non-anticipating stochastic derivative introduced in [15]. We prove both a sufficient and necessary stochastic maximum principle.
The general maximum principle is proved for an infinite dimensional controlled stochastic evolution system. The control is allowed to take values in a nonconvex set and enter into both drift and diffusion terms. The operator-valued backward stochastic differential equation, which characterizes the second-order adjoint process, is understood via the concept of "generalized solution" proposed by Guatteri and Tessitore [SICON 44 (2006)].
It is well-known from the work of Schönbucher (2005) that the marginal laws of a loss process can be matched by a unit increasing time inhomogeneous Markov process, whose deterministic jump intensity is called local intensity. The Stochastic Local Intensity (SLI) models such as the one proposed by Arnsdorf and Halperin (2008) allow to get a stochastic jump intensity while keeping the same marginal laws. These models involve a non-linear SDE with jumps. The first contribution of this paper is to prove the existence and uniqueness of such processes. This is made by means of an interacting partic…
We investigate a stochastic optimal control problem where the controlled system is depicted as a stochastic differential delayed equation; however, at the terminal time, the state is constrained in a convex set. We firstly introduce an equivalent backward delayed system depicted as a time-delayed backward stochastic differential equation. Then a stochastic maximum principle is obtained by virtue of Ekeland's variational principle. Finally, applications to a state constrained stochastic delayed linear-quadratic control model and a production-consumption choice problem are studied to illustrate …
This paper concerns the parameter estimation problem for the quadratic potential energy in interacting particle systems from continuous-time and single-trajectory data. Even though such dynamical systems are high-dimensional, we show that the vanilla maximum likelihood estimator (without regularization) is able to estimate the interaction potential parameter with optimal rate of convergence simultaneously in mean-field limit and in long-time dynamics. This to some extend avoids the curse-of-dimensionality for estimating large dynamical systems under symmetry of the particle interaction.
Stochastic maximum principle for time-changed forward-backward stochastic control problem with Lévy noise
Yau Mathematical Sciences Center, Tsinghua University（丘成桐数学科学中心，清华大学）；Department of Mathematical Sciences, Tsinghua University（数学科学系，清华大学）；Department of Automation, Center for Brain-Inspired Computing Research, Tsinghua University（自动化系，脑启发计算研究中心，清华大学）
We study a stochastic differential game between two players, controlling a forward stochastic Volterra integral equation (FSVIE). Each player has to optimize his own performance functional which includes a backward stochastic differential equation (BSDE). The dynamics considered are driven by time-changed Lévy noises, with absolutely continuous time-change process. We prove a sufficient maximum principle to characterize Nash equilibria and the related optimal strategies. For this we use techniques of control under partial information, and the non-anticipating stochastic derivative. The zero-su…
In this paper, we prove both necessary and sufficient maximum principles for infinite horizon discounted control problems of stochastic Volterra integral equations with finite delay and a convex control domain. The corresponding adjoint equation is a novel class of infinite horizon anticipated backward stochastic Volterra integral equations. Our results can be applied to discounted control problems of stochastic delay differential equations and fractional stochastic delay differential equations. As an example, we consider a stochastic linear-quadratic regulator problem for a delayed fractional…
We study the optimal control problem for a weighted mean-field system. A new feature of the control problem is that the coefficients depend on the state process as well as its weighted measure and the control variable. By applying variational technique, we establish a stochastic maximum principle. As an application, we investigate the optimal premium policy of an insurance firm for asset-liability management problem.
'Rich get richer' rule comforts previously often chosen actions. What is happening to the evolution of individual inclinations to choose an action when agents do interact ? Interaction tends to homogenize while each individual dynamics tends to reinforce its own position. Interacting stochastic systems of reinforced processes were recently considered in many papers, where the asymptotic behavior was proven to exhibit a.s. synchronization. We consider in this paper models where, even if interaction among agents is present, absence of synchronization may happen due to the choice of an individual…
This paper studies a time-changed stochastic control problem, where the underlying stochastic process is a Lévy noise time-changed by an inverse subordinator. We establish a maximum principle theory for the time-changed stochastic control problem. We also prove the existence and uniqueness of the corresponding time-changed backward stochastic differential equation involved in the stochastic control problem. Some examples are provided for illustration.
This paper formulates and studies a stochastic maximum principle for forward-backward stochastic Volterra integral equations (FBSVIEs in short), while the control area is assumed to be convex. Then a linear quadratic (LQ in short) problem for backward stochastic Volterra integral equations (BSVIEs in short) is present to illustrate the aforementioned optimal control problem. Motivated by the technical skills in solving above problem, a more convenient and briefer method for the unique solvability of M-solution for BSVIEs is proposed. At last, we will investigate a risk minimization problem by …
We consider a stochastic control problem where the set of controls is not necessarily convex and the system is governed by a nonlinear backward stochastic differential equation. We establish necessary as well as sufficient conditions of optimality for two models. The first concerns the strict (classical) controls. The second is an extension of the first to relaxed controls, who are a measure valued processes.
The main purpose of this paper is to give a solution to a long-standing unsolved problem in stochastic control theory, i.e., to establish the Pontryagin-type maximum principle for optimal controls of general infinite dimensional nonlinear stochastic evolution equations. Both drift and diffusion terms can contain the control variables, and the control domains are allowed to be nonconvex. The key to reach it is to provide a suitable formulation of operator-valued backward stochastic evolution equations (BSEEs for short), as well as a way to define their solutions. Besides, both vector-valued and…
In this paper we prove a version of the maximum principle, in the sense of Pontryagin, for the optimal control of a finite dimensional stochastic differential equation, driven by a multidimensional Wiener process. We drop the usual Lipschitz assumption on the drift term and substitute it with dissipativity conditions, allowing polynomial growth. The control enter both the drift and the diffusion term and takes values in a general metric space.
Stochastic maximum principle of nonlinear controlled forward-backward systems, where the set of strict (classical) controls need not be convex and the diffusion coefficient depends explicitly on the variable control, is an open problem impossible to solve by the classical method of spike variation. In this paper, we introduce a new approach to solve this open problem and we establish necessary as well as sufficient conditions of optimality, in the form of global stochastic maximum principle, for two models. The first concerns the relaxed controls, who are a measure-valued processes. The second…
We solve infinite-dimensional stochastic differential equations (ISDEs) describing an infinite number of Brownian particles interacting via two-dimensional Coulomb potentials. The equilibrium states of the associated unlabeled stochastic dynamics are the Ginibre random point field and Dyson's measures, which appear in random matrix theory. To solve the ISDEs we establish an integration by parts formula for these measures. Because the long-range effect of two-dimensional Coulomb potentials is quite strong, the properties of Brownian particles interacting with two-dimensional Coulomb potentials …
We discuss maximum likelihood estimation of parameters for models governed by a stochastic differential equation driven by a mixed fractional Brownian motion with random effects.
This paper considers a controlled Itô-Lévy process where the information available to the controller is possibly less than the overall information. All the system coefficients and the objective performance functional are allowed to be random, possibly non-Markovian. Malliavin calculus is employed to derive a maximum principle for the optimal control of such a system where the adjoint process is explicitly expressed.
We consider in this paper, mixed relaxed-singular stochastic control problems, where the control variable has two components, the first being measure-valued and the second singular. The control domain is not necessarily convex and the system is governed by a nonlinear stochastic differential equation, in which the measure-valued part of the control enters both the drift and the diffusion coefficients. We establish necessary optimality conditions, of the Pontryagin maximum principle type, satisfied by an optimal relaxed-singular control, which exist under general conditions on the coefficients.…
We extend Peng's maximum principle for semilinear stochastic partial differential equations (SPDEs) in one space-dimension with non-convex control domains and control-dependent diffusion coefficients to the case of general cost functionals with Nemytskii-type coefficients. Our analysis is based on a new approach to the characterization of the second order adjoint state as the solution of a function-valued backward SPDE.
This paper is based on the talk in "Probability Symposium" at Research Institute of Mathematical Sciences (Kyoto University) on 2013/12/18, and gives an announcement of some parts of the results in [1,8,10,11]. We show two instances of dynamical rigidity of Ginibre interacting Brownian motion in infinite dimensions. This stochastic dynamics is given by the infinite-dimensional stochastic differential equation describing infinite-many Brownian particles in the plane interacting through two-dimensional Coulomb potential. The first dynamical rigidity is that the Ginibre interacting Brownian motio…
