We consider the scheduling control problem for a family of unitary networks under heavy traffic, with general interarrival and service times, probabilistic routing and infinite horizon discounted linear holding cost. A natural nonanticipativity condition for admissibility of control policies is introduced. The condition is seen to hold for a broad class of problems. Using this formulation of admissible controls and a time-transformation technique, we establish that the infimum of the cost for the network control problem over all admissible sequencing control policies is asymptotically bounded below by the value function of an associated diffusion control problem (the Brownian control problem). This result provides a useful bound on the best achievable performance for any admissible control policy for a wide class of networks.
In this note, we propose two different approaches to rigorously justify a pseudo-Markov property for controlled diffusion processes which is often (explicitly or implicitly) used to prove the dynamic programming principle in the stochastic control literature. The first approach develops a sketch of proof proposed by Fleming and Souganidis~\cite{fleming-souganidis}. The second approach is based on an enlargement of the original state space and a controlled martingale problem. We clarify some measurability and topological issues raised by these two approaches.
We consider a controlled second order differential equation which is partially observed with an additional fractional noise. we study the asymptotic (for large observation time) design problem of the input and give an efficient estimator of the unknown signal drift parameter. When the input depends on the unknow parameter, we will try the one-step estimation procedure using the Newton-Raphson method.
This article gives an overview of the developments in controlled diffusion processes, emphasizing key results regarding existence of optimal controls and their characterization via dynamic programming for a variety of cost criteria and structural assumptions. Stochastic maximum principle and control under partial observations (equivalently, control of nonlinear filters) are also discussed. Several other related topics are briefly sketched.
We consider a rate control problem for an $N$-particle weakly interacting finite state Markov process. The process models the state evolution of a large collection of particles and allows for multiple particles to change state simultaneously. Such models have been proposed for large communication systems (e.g. ad hoc wireless networks) but are also suitable for other settings such as chemical-reaction networks. An associated diffusion control problem is presented and we show that the value function of the $N$-particle controlled system converges to the value function of the limit diffusion con…
In this paper, we study the asymptotic of exit problem for controlled Markov diffusion processes with random jumps and vanishing diffusion terms, where the random jumps are introduced in order to modify the evolution of the controlled diffusions by switching from one mode of dynamics to another. That is, depending on the state-position and state-transition information, the dynamics of the controlled diffusions randomly switches between the different drift and diffusion terms. Here, we specifically investigate the asymptotic exit problem concerning such controlled Markov diffusion processes in …
We study stochastic delay differential equations (SDDE) where the coefficients depend on the moving averages of the state process. As a first contribution, we provide sufficient conditions under which a linear path functional of the solution of a SDDE admits a finite-dimensional Markovian representation. As a second contribution, we show how approximate finite-dimensional Markovian representations may be constructed when these conditions are not satisfied, and provide an estimate of the error corresponding to these approximations. These results are applied to optimal control and optimal stoppi…
We investigate the asymptotic behavior of the a large class of reversible chemical reaction-diffusion equations with the same diffusion. In particular we prove the optimal rate in two cases : when there is no diffusion and in the classical "two-by-two" case.
We study a family of mean field games with a state variable evolving as a multivariate jump diffusion process. The jump component is driven by a Poisson process with a time-dependent intensity function. All coefficients, i.e. drift, volatility and jump size, are controlled. Under fairly general conditions, we establish existence of a solution in a relaxed version of the mean field game and give conditions under which the optimal strategies are in fact Markovian, hence extending to a jump-diffusion setting previous results established in [30]. The proofs rely upon the notions of relaxed control…
The main result in this paper is a variational formula for the exit rate from a bounded domain for a diffusion process in terms of the stationary law of the diffusion constrained to remain in this domain forever. Related results on the geometric ergodicity of the controlled Q-process are also presented.
We continue the approach in Part I \cite{duchong19} to study stationary states of controlled differential equations driven by rough paths, using the framework of random dynamical systems and random attractors. Part II deals with driving paths of finite $ν$ - Hölder norms with $ν\in (\frac{1}{3},\frac{1}{2})$ so that the integrals are interpreted in the Gubinelli sense for controlled rough paths. We prove sufficient conditions for the attractor to be a singleton, thus the pathwise convergence is in both pullback and forward senses.
We present precise asymptotics of the odometer function for the internal Diffusion Limited Aggregation model. These results provide a better understanding of this function whose importance was demonstrated by Levine and Peres. We derive a different proof of a time-scale result by Lawler, Bramson and Griffeath.
Is studied asymptotic expansion for solution of singularly perturbed equation for functional of Markovian evolution in Rd. The view of regular and singular parts of solution is found.
We consider a mean field game with common noise in which the diffusion coefficients may be controlled. We prove existence of a weak relaxed solution under some continuity conditions on the coefficients. We then show that, when there is no common noise, the solution of this mean field game is characterized by a McKean-Vlasov type second order backward SDE.
Time-inhomogeneous controlled diffusion processes in both cylindrical and noncylindrical domains are considered. Bellman's principle and its applications to proving the continuity of value functions are investigated.
We study a one-dimensional diffusion process in a drifted Brownian potential. We characterize the upper functions of its hitting times in the sense of Paul Lévy, and determine the lower limits in terms of an iterated logarithm law.
The uniqueness argument in the proof of Theorem 5, p. 483, of "Small noise asymptotics for invariant densities for a class of diffusions: a control theoretic view, J. Math. Anal. and Appl. (2009) " is flawed. We give here a corrected proof.
This paper establishes the global asymptotic equivalence, in the sense of the Le Cam $Δ$-distance, between scalar diffusion models with unknown drift function and small variance on the one side, and nonparametric autoregressive models on the other side. The time horizon $T$ is kept fixed and both the cases of discrete and continuous observation of the path are treated. We allow non constant diffusion coefficient, bounded but possibly tending to zero. The asymptotic equivalences are established by constructing explicit equivalence mappings.
We provide Large Deviation estimates for the bridge of a $d$-dimensional general diffusion process as the conditioning time tends to $0$ and apply these results to the evaluation of the asymptotics of its exit time probabilities. We are motivated by applications to numerical simulation, especially in connection with stochastic volatility models.
We provide a unified analytic approach to study stationary states of controlled differential equations driven by rough paths, using the framework of random dynamical systems and random attractors. Part I deals with driving paths of finite $p$-variations with $1 \leq p <2$ so that the integrals are interpreted in the Young sense. Our method helps to generalize recent results \cite{GAKLBSch2010}, \cite{ducGANSch18}, \cite{duchongcong18} on the existence of the global pullback attractors for the generated random dynamical systems. We also prove sufficient conditions for the attractor to be a sing…
In [8], asymptotic expansion of the martingale with mixed normal limit was provided. The expansion formula is expressed by the adjoint of a random symbol with coefficients described by the Malliavin calculus, differently from the standard invariance principle. As an application, an asymptotic expansion for a quadratic form of a diffusion process was derived in the same paper. This article gives some details of the derivation, after a short review of the martingale expansion in mixed normal limit.
We study a class of non linear integro-differential equations on the Wasserstein space related to the optimal control of McKean--Vlasov jump-diffusions. We develop an intrinsic notion of viscosity solutions that does not rely on the lifting to an Hilbert space and prove a comparison theorem for these solutions. We also show that the value function is the unique viscosity solution.
In this paper, we discuss long-time behavior of sample paths for a wide range of regime-switching diffusions. Firstly, almost sure asymptotic stability is concerned (i) for regime-switching diffusions with finite state spaces by the Perron-Frobenius theorem, and, with regard to the case of reversible Markov chain, via the principal eigenvalue approach; (ii) for regime-switching diffusions with countable state spaces by means of a finite partition trick and an M-Matrix theory. We then apply our theory to study the stabilization for linear switching models. Several examples are given to demonstr…
We obtain non-asymptotic Gaussian concentration bounds for the difference between the invariant measure $ν$ of an ergodic Brownian diffusion process and the empirical distribution of an approximating scheme with decreasing time step along a suitable class of (smooth enough) test functions f such that f -- $ν$(f) is a coboundary of the infinitesimal generator. We show that these bounds can still be improved when the (squared) Fr{ö}benius norm of the diffusion coefficient lies in this class. We apply these bounds to design computable non-asymptotic confidence intervals for the approximating sche…
This paper considers the limiting distribution of $π_{λ,θ}$, the stationary distribution of the infinitely-many-alleles diffusion with symmetric overdominance \cite{MR1626158}. In \cite{MR2519357} the large deviation principle for $π_{λ,θ}$ indicates that there are countably many phase transitions for the limiting distribution of $π_{λ,θ}$, and the critical points are $λ=k(k+1), k\geq1$. The asymptotic behaviours at those critical points, however, are unclear. This article provides a definite description of the critical cases.
We address the variational formulation of the risk-sensitive reward problem for non-degenerate diffusions on $\mathbb{R}^d$ controlled through the drift. We establish a variational formula on the whole space and also show that the risk-sensitive value equals the generalized principal eigenvalue of the semilinear operator. This can be viewed as a controlled version of the variational formulas for principal eigenvalues of diffusion operators arising in large deviations. We also revisit the average risk-sensitive minimization problem and by employing a gradient estimate developed in this paper, w…
We investigate the long time behavior of weakly dissipative semilinear Hamilton-Jacobi-Bellman (HJB) equations and the turnpike property for the corresponding stochastic control problems. To this aim, we develop a probabilistic approach based on a variant of coupling by reflection adapted to the study of controlled diffusion processes. We prove existence and uniqueness of solutions for the ergodic Hamilton-Jacobi-Bellman equation and different kind of quantitative exponential convergence results at the level of the value function, of the optimal controls and of the optimal processes. Moreover,…
We consider time-changed diffusions driven by generators with discontinuous coefficients. The PDE's connections are investigated and in particular some results on the asymptotic analysis according to the behaviour of the coefficients are presented.
A dynamical system perturbed by white noise in a neighborhood of an unstable fixed point is considered. We obtain the exit asymptotics in the limit of vanishing noise intensity. This is a refinement of a result by Kifer (1981).
In this paper, we develop asymptotic theory for the mixing detection methodology proposed by M. Magdziarz and A. Weron [Physical Review E, 84:051138 (2011)]. The assumptions cover a broad family of Gaussian stochastic processes including fractional Gaussian noise and the fractional Ornstein-Uhlenbeck process. We show that the asymptotic distribution and convergence rates of the detection statistic may be, respectively, Gaussian or non-Gaussian and standard or nonstandard depending on the diffusion exponent. The results pave the way for mixing detection based on a single observed sample path an…
We consider the asymptotic behaviour of the solution of one dimensional stochastic differential equations and Langevin equations in periodic backgrounds with zero average. We prove that in several such models, there is generically a non vanishing asymptotic velocity, despite of the fact that the average of the background is zero.
This paper demonstrates a lower and upper solution method to investigate the asymptotic behaviour of the conservative reaction-diffusion systems associated with Markovian process algebra models. In particular, we have proved the uniform convergence of the solution to its constant equilibrium for a case study as time tends to infinity, together with experimental results illustrations.
We consider triangular arrays of Markov chains that converge weakly to a diffusion process. Local limit theorems for transition densities are proved. The observation time [0,T] may be fixed or lim n T = 0, where nh = T and h is a mesh between two neighboring observation points.
Universität Potsdam（波茨坦大学）；Freie Universität Berlin（柏林自由大学）；Brandenburgische Technische Universität Cottbus-Senftenberg（勃兰登堡工业大学科特布斯-森夫滕贝格）
Optimal control of diffusion processes is intimately connected to the problem of solving certain Hamilton-Jacobi-Bellman equations. Building on recent machine learning inspired approaches towards high-dimensional PDEs, we investigate the potential of $\textit{iterative diffusion optimisation}$ techniques, in particular considering applications in importance sampling and rare event simulation, and focusing on problems without diffusion control, with linearly controlled drift and running costs that depend quadratically on the control. More generally, our methods apply to nonlinear parabolic PDEs…
We determine the long-time asymptotic behavior of a relativistic diffusion taking values in the unitary tangent bundle of a Robertson-Walker space-time. We prove in particular that when approaching the explosion time of the diffusion, its projection on the base manifold almost surely converges to a random point of the causal boundary and we also describe the behavior of the tangent vector in the neighborhood of this limiting point.
We study asymptotic properties of conditional least squares estimators for the drift parameters of two-factor affine diffusions based on continuous time observations. We distinguish three cases: subcritical, critical and supercritical. For all the drift parameters, in the subcritical and supercritical cases, asymptotic normality and asymptotic mixed normality is proved, while in the critical case, non-standard asymptotic behavior is described.
