This article gives an overview of the developments in controlled diffusion processes, emphasizing key results regarding existence of optimal controls and their characterization via dynamic programming for a variety of cost criteria and structural assumptions. Stochastic maximum principle and control under partial observations (equivalently, control of nonlinear filters) are also discussed. Several other related topics are briefly sketched.
In this note, we propose two different approaches to rigorously justify a pseudo-Markov property for controlled diffusion processes which is often (explicitly or implicitly) used to prove the dynamic programming principle in the stochastic control literature. The first approach develops a sketch of proof proposed by Fleming and Souganidis~\cite{fleming-souganidis}. The second approach is based on an enlargement of the original state space and a controlled martingale problem. We clarify some measurability and topological issues raised by these two approaches.
We consider a controlled second order differential equation which is partially observed with an additional fractional noise. we study the asymptotic (for large observation time) design problem of the input and give an efficient estimator of the unknown signal drift parameter. When the input depends on the unknow parameter, we will try the one-step estimation procedure using the Newton-Raphson method.
We consider a rate control problem for an $N$-particle weakly interacting finite state Markov process. The process models the state evolution of a large collection of particles and allows for multiple particles to change state simultaneously. Such models have been proposed for large communication systems (e.g. ad hoc wireless networks) but are also suitable for other settings such as chemical-reaction networks. An associated diffusion control problem is presented and we show that the value function of the $N$-particle controlled system converges to the value function of the limit diffusion con…
In this paper, a special sequence of controlled branching processes is considered. We provide a simple set of sufficient conditions for the weak convergence of such processes to a weak solution to a kind of continuous branching processes with dependent immigration.
This survey aims at collecting and presenting results for one-type, discrete time branching processes with random control functions. In particular, the subclass of critical migration processes with different regimes of immigration and emigration is reviewed in detail. Critical controlled branching processes with continuous state space are also discussed.
In this paper, we study the asymptotic of exit problem for controlled Markov diffusion processes with random jumps and vanishing diffusion terms, where the random jumps are introduced in order to modify the evolution of the controlled diffusions by switching from one mode of dynamics to another. That is, depending on the state-position and state-transition information, the dynamics of the controlled diffusions randomly switches between the different drift and diffusion terms. Here, we specifically investigate the asymptotic exit problem concerning such controlled Markov diffusion processes in …
We study stochastic delay differential equations (SDDE) where the coefficients depend on the moving averages of the state process. As a first contribution, we provide sufficient conditions under which a linear path functional of the solution of a SDDE admits a finite-dimensional Markovian representation. As a second contribution, we show how approximate finite-dimensional Markovian representations may be constructed when these conditions are not satisfied, and provide an estimate of the error corresponding to these approximations. These results are applied to optimal control and optimal stoppi…
We study a family of mean field games with a state variable evolving as a multivariate jump diffusion process. The jump component is driven by a Poisson process with a time-dependent intensity function. All coefficients, i.e. drift, volatility and jump size, are controlled. Under fairly general conditions, we establish existence of a solution in a relaxed version of the mean field game and give conditions under which the optimal strategies are in fact Markovian, hence extending to a jump-diffusion setting previous results established in [30]. The proofs rely upon the notions of relaxed control…
In this paper, we prove that the laws of perturbed diffusion processes and perturbed reflected diffusion processes are absolutely continuous with respect to the Lebesgue measure. The main tool we use is the Malliavin calculus.
The main result in this paper is a variational formula for the exit rate from a bounded domain for a diffusion process in terms of the stationary law of the diffusion constrained to remain in this domain forever. Related results on the geometric ergodicity of the controlled Q-process are also presented.
We consider a mean field game with common noise in which the diffusion coefficients may be controlled. We prove existence of a weak relaxed solution under some continuity conditions on the coefficients. We then show that, when there is no common noise, the solution of this mean field game is characterized by a McKean-Vlasov type second order backward SDE.
Time-inhomogeneous controlled diffusion processes in both cylindrical and noncylindrical domains are considered. Bellman's principle and its applications to proving the continuity of value functions are investigated.
We study a class of non linear integro-differential equations on the Wasserstein space related to the optimal control of McKean--Vlasov jump-diffusions. We develop an intrinsic notion of viscosity solutions that does not rely on the lifting to an Hilbert space and prove a comparison theorem for these solutions. We also show that the value function is the unique viscosity solution.
In this note, we consider the necessary and sufficient condition for viability property of diffusion processes with jumps on closed submanifolds of $R^{m}$ with some concrete examples.
In this paper we investigate the long time behavior of a family of diffusion processes with Hölder continuous diffusion terms on a compact set, these process arise naturally in random approximations of an ODE. We will prove that these processes hit the boundary in finite time and prove the existence of a quasi-stationnary distribution
We provide some on-off type criteria for recurrence and transience of regime-switching diffusion processes using the theory of M-matrix and the Perron-Frobenius theorem. State-independent and state-dependent regime-switching diffusion processes in a finite space and a countable space are both studied. We put forward a finite partition method to deal with switching process in a countable space. As an application, we improve the known criteria for recurrence of linear regime-switching diffusion processes, and provide an on-off type criterion for a kind of nonlinear regime-switching diffusion pro…
In this paper a concentration inequality is proved for the deviation in the ergodic theorem in the case of discrete time observations of diffusion processes. The proof is based on the geometric ergodicity property for diffusion processes. As an application we consider the nonparametric pointwise estimation problem for the drift coefficient under discrete time observations.
In this paper, we introduce a new method of sampling from transition densities of diffusion processes including those unknown in closed forms by solving a partial differential equation satisfied by the quotient of transition densities. We demonstrate the performance of the developed method on processes with known densities and the obtained results are consistent with theoretical values. The method is applied to Wright-Fisher diffusions owing to their importance in population genetics in studying interaction networks inherent in genetic data. Diffusion processes with bounded drift and non degen…
In this paper, we introduce an index which measures the strength of recurrence of symmetric Markov processes, and give some sufficient conditions for recurrence of direct products of symmetric diffusion processes. The index is given by the Dirichlet forms of the Markov processes. Moreover, as an application, we prove the recurrence of some multi-dimensional diffusion processes in random environments including zero potentials.
The spectral gap is estimated for measure-valued diffusion processes induced by the intrinsic/extrinsic derivatives on the space of finite measures over a Riemannian manifold. This provides explicit exponential convergence rate for these processes to approximate the Dirichlet and Gamma distributions arising from population genetics.
We provide some criteria on the stability of regime-switching diffusion processes. Both the state-independent and state-dependent regime-switching diffusion processes with switching in a finite state space and an infinite countable state space are studied in this work. We provide two methods to deal with switching processes in an infinite countable state space. One is a finite partition method based on the nonsingular M-matrix theory. Another is an application of principal eigenvalue of a bilinear form. Our methods can deal with both linear and nonlinear regime-switching diffusion processes. M…
We consider the scheduling control problem for a family of unitary networks under heavy traffic, with general interarrival and service times, probabilistic routing and infinite horizon discounted linear holding cost. A natural nonanticipativity condition for admissibility of control policies is introduced. The condition is seen to hold for a broad class of problems. Using this formulation of admissible controls and a time-transformation technique, we establish that the infimum of the cost for the network control problem over all admissible sequencing control policies is asymptotically bounded …
We investigate the existence and uniqueness of strong solutions up to an explosion time for regime-switching diffusion processes in an infinite state space. Instead of concrete conditions on coefficients, our existence and uniqueness result is established under the general assumption that the diffusion in every fixed environment has a unique non-explosive strong solution. Moreover, non-explosion conditions for regime-switching diffusion processes are given. The strong Feller property is proved by further assuming that the diffusion in every fixed environment generates a strong Feller semigroup…
A simple nonlinear integral equation for Ito's map is obtained. Although, it does not include stochastic integrals, it does give causal construction of diffusion processes which can be easily implemented by iteration systems. Applications in financial modelling and extension to fBm are discussed.
Given a continuous Gaussian process $x$ which gives rise to a $p$-geometric rough path for $p\in (2,3)$, and a general continuous process $y$ controlled by $x$, under proper conditions we establish the relationship between the Skorohod integral $\int_0^t y_s {\mathrm{d}}^\diamond x_s$ and the Stratonovich integral $\int_0^t y_s {\mathrm{d}} {\mathbf x}_s$. Our strategy is to employ the tools from rough paths theory and Malliavin calculus to analyze discrete sums of the integrals.
We establish the existence and pathwise uniqueness of regime-switching diffusion processes in an infinite state space, which could be time-inhomogeneous and state-dependent. Then the strong Feller properties of these processes are investigated by using the theory of parabolic differential equations and dimensional-free Harnack inequalities.
Central limit theorems play an important role in the study of statistical inference for stochastic processes. However, when the nonparametric local polynomial threshold estimator, especially local linear case, is employed to estimate the diffusion coefficients of diffusion processes, the adaptive and predictable structure of the estimator conditionally on the $σ-$field generated by diffusion processes is destroyed, the classical central limit theorem for martingale difference sequences can not work. In this paper, we proved the central limit theorems of local polynomial threshold estimators fo…
For n-dimensional ergodic diffusion processes with values in $G=\mathbb{R}_{+}^n$ we prove time-independent upper bounds for the transitional density and so also for the unique ergodic density. We do not require geodesic completeness of the elliptic symbol towards the boundary of $G$.
This work focuses on a class of regime-switching jump diffusion processes, in which the switching component has countably infinite many states or regimes. The existence and uniqueness of the underlying process are obtained by an interlacing procedure. Then the Feller and strong Feller properties of such processes are derived by the coupling method and an appropriate Radon-Nikodym derivative. Finally the paper studies exponential ergodicity of regime-switching jump-diffusion processes.
This paper is devoted to the study of the following problem. We have set of diffusion processes with absorption on boundaries in some region at initial time $t=0$. It is required to estimate of number of the unabsorbed processes for the fixed time ~$τ>0$. The number of initial processes is considered as function of $τ$ and $τ\to\infty$.
We study jump-diffusion processes with parameters switching at random times. Being motivated by possible applications, we characterise equivalent martingale measures for these processes by means of the relative entropy. The minimal entropy approach is also developed. It is shown that in contrast to the case of Lévy processes, for this model an Esscher transformation does not produce the minimal relative entropy.
We have random number of independent diffusion processes with absorption on boundaries in some region at initial time $t=0$. The initial numbers and positions of processes in region is defined by Poisson random measure. It is required to estimate of number of the unabsorbed processes for the fixed time \~$τ>0$. The Poisson random measure depends on $τ$ and $τ\to\infty$.
We present a central limit theorem for stationary random fields that are short-range dependent and asymptotically independent. As an application, we present a central limit theorem for an infinite family of interacting Itô-type diffusion processes.
Since T. Lyons invented rough path theory, one of its most successful applications is a new proof of Freidlin-Wentzell's large deviation principle for diffusion processes. In this paper we extend this method to the case of pinned diffusion processes under a mild ellipticity assumption. Besides rough path theory, our main tool is quasi-sure analysis, which is a kind of potential theory in Malliavin calculus.
We address the variational formulation of the risk-sensitive reward problem for non-degenerate diffusions on $\mathbb{R}^d$ controlled through the drift. We establish a variational formula on the whole space and also show that the risk-sensitive value equals the generalized principal eigenvalue of the semilinear operator. This can be viewed as a controlled version of the variational formulas for principal eigenvalues of diffusion operators arising in large deviations. We also revisit the average risk-sensitive minimization problem and by employing a gradient estimate developed in this paper, w…
We investigate the long time behavior of weakly dissipative semilinear Hamilton-Jacobi-Bellman (HJB) equations and the turnpike property for the corresponding stochastic control problems. To this aim, we develop a probabilistic approach based on a variant of coupling by reflection adapted to the study of controlled diffusion processes. We prove existence and uniqueness of solutions for the ergodic Hamilton-Jacobi-Bellman equation and different kind of quantitative exponential convergence results at the level of the value function, of the optimal controls and of the optimal processes. Moreover,…
We lead the algorithm of expansion of sojourn probability of many-dimensional diffusion processes in small domain. The principal member of this expansion defines normalizing coefficient for special limit theorems.
We consider diffusion processes in Hilbert spaces with constant non-degenerate diffusion operators and show that, under broad assumptions on the drift, the transition probabilities of the process are positive on ellipsoids associated with the diffusion operator. This is an infinite-dimensional analogue of positivity of densities of transition probabilities. Our results apply to diffusions corresponding to stochastic partial differential equations.
We a controlled system driven by a coupled forward-backward stochastic differential equation (FBSDE) with a non degenerate diffusion matrix. The cost functional is defined by the solution of the controlled backward stochastic differential equation (BSDE), at the initial time. Our goal is to find an optimal control which minimizes the cost functional. The method consists to construct a sequence of approximating controlled systems for which we show the existence of a sequence of feedback optimal controls. By passing to the limit, we establish the existence of a relaxed optimal control to the ini…
Optimal control problems are inherently hard to solve as the optimization must be performed simultaneously with updating the underlying system. Starting from an initial guess, Howard's policy improvement algorithm separates the step of updating the trajectory of the dynamical system from the optimization and iterations of this should converge to the optimal control. In the discrete space-time setting this is often the case and even rates of convergence are known. In the continuous space-time setting of controlled diffusion the algorithm consists of solving a linear PDE followed by maximization…
We aim at characterizing viability, invariance and some reachability properties of controlled piecewise deterministic Markov processes (PDMPs). Using analytical methods from the theory of viscosity solutions, we establish criteria for viability and invariance in terms of the first order normal cone. We also investigate reachability of arbitrary open sets. The method is based on viscosity techniques and duality for some associated linearized problem. The theoretical results are applied to general On/Off systems, Cook's model for haploinssuficiency, and a stochastic model for bacteriophage lambd…
The paper studies the rate of convergence of the weak Euler approximation for Itô diffusion and jump processes with Hölder-continuous generators. It covers a number of stochastic processes including the nondegenerate diffusion processes and a class of stochastic differential equations driven by stable processes. To estimate the rate of convergence, the existence of a unique solution to the corresponding backward Kolmogorov equation in Hölder space is first proved. It then shows that the Euler scheme yields positive weak order of convergence.
This paper presents new results on the Edgeworth expansion for high frequency functionals of continuous diffusion processes. We derive asymptotic expansions for weighted functionals of the Brownian motion and apply them to provide the second order Edgeworth expansion for power variation of diffusion processes. Our methodology relies on martingale embedding, Malliavin calculus and stable central limit theorems for semimartingales. Finally, we demonstrate the density expansion for studentized statistics of power variations.
Universität Potsdam（波茨坦大学）；Freie Universität Berlin（柏林自由大学）；Brandenburgische Technische Universität Cottbus-Senftenberg（勃兰登堡工业大学科特布斯-森夫滕贝格）
Optimal control of diffusion processes is intimately connected to the problem of solving certain Hamilton-Jacobi-Bellman equations. Building on recent machine learning inspired approaches towards high-dimensional PDEs, we investigate the potential of $\textit{iterative diffusion optimisation}$ techniques, in particular considering applications in importance sampling and rare event simulation, and focusing on problems without diffusion control, with linearly controlled drift and running costs that depend quadratically on the control. More generally, our methods apply to nonlinear parabolic PDEs…
In this paper we derive explicit formulas of the Rényi information, Shannon entropy and Song measure for the invariant density of one dimensional ergodic diffusion processes. In particular, the diffusion models considered include the hyperbolic, the generalized inverse Gaussian, the Pearson, the exponential familiy and a new class of skew-$t$ diffusions.
It is given an effective upper estimate of expectation of |T_1-T_2|, where T_1 and T_2 are the first exit times from a region for two vector diffusion processes.
Set Invariance with Probability One for Controlled Diffusion: Score-based Approach
Department of Aerospace Engineering, Iowa State University（航空航天工程系，爱荷华州立大学）；Department of Applied Mathematics, University of California, Santa Cruz（应用数学系，加州大学圣克鲁兹分校）；Department of Electrical Engineering and Computer Sciences, University of California, Berkeley（电气工程与计算机科学系，加州大学伯克利分校）
