We consider controlled martingales with bounded steps where the controller is allowed at each step to choose the distribution of the next step, and where the goal is to hit a fixed ball at the origin at time $n$. We show that the algebraic rate of decay (as $n$ increases to infinity) of the value function in the discrete setup coincides with its continuous counterpart, provided a reachability assumption is satisfied. We also study in some detail the uniformly elliptic case and obtain explicit bounds on the rate of decay. This generalizes and improves upon several recent studies of the one dimensional case, and is a discrete analogue of a stochastic control problem recently investigated in Armstrong and Trokhimtchouck [Calc. Var. Partial Differential Equations 38 (2010) 521-540].
We provide non-asymptotic convergence rates of the Polyak-Ruppert averaged stochastic gradient descent (SGD) to a normal random vector for a class of twice-differentiable test functions. A crucial intermediate step is proving a non-asymptotic martingale central limit theorem (CLT), i.e., establishing the rates of convergence of a multivariate martingale difference sequence to a normal random vector, which might be of independent interest. We obtain the explicit rates for the multivariate martingale CLT using a combination of Stein's method and Lindeberg's argument, which is then used in conjun…
We provide verification theorems (at different levels of generality) for infinite horizon stochastic control problems in continuous time for semimartingales. The control framework is given as an abstract "martingale formulation", which encompasses a broad range of standard control problems. Under appropriate conditions we show that the set of admissible controls gives rise to a certain class of controlled special semimartingales. Our results generalise both the standard controlled Itô- and Lévy-diffusion settings as we allow ourselves to locally control not only the drift and diffusion coeffic…
This paper is devoted to a direct martingale approach for P{ó}lya urn models asymptotic behaviour. A P{ó}lya process is said to be small when the ratio of its remplacement matrix eigenvalues is less than or equal to 1/2, otherwise it is called large. We find again some well-known results on the asymptotic behaviour for small and large urns processes. We also provide new almost sure properties for small urns processes.
Constrained Markov processes, such as reflecting diffusions, behave as an unconstrained process in the interior of a domain but upon reaching the boundary are controlled in some way so that they do not leave the closure of the domain. In this paper, the behavior in the interior is specified by a generator of a Markov process, and the constraints are specified by a controlled generator. Together, the generators define a "constrained martingale problem". The desired constrained processes are constructed by first solving a simpler "controlled martingale problem" and then obtaining the desired pro…
This paper is devoted to the asymptotic analysis of the reinforced elephant random walk (RERW) using a martingale approach. In the diffusive and critical regimes, we establish the almost sure convergence, the law of iterated logarithm and the quadratic strong law for the RERW. The distributional convergences of the RERW to some Gaussian processes are also provided. In the superdiffusive regime, we prove the distributional convergence as well as the mean square convergence of the RERW. All our analysis relies on asymptotic results for multi-dimensional martingales with matrix normalization.
Markovian growth-fragmentation processes introduced by Bertoin extend the pure fragmentation model by allowing the fragments to grow larger or smaller between dislocation events. What becomes of the known asymptotic behaviors of self-similar pure fragmentations when growth is added to the fragments is a natural question that we investigate in this paper. Our results involve the terminal value of some additive martingales whose uniform integrability is an essential requirement. Dwelling first on the homogeneous case, we exploit the connection with branching random walks and in particular the ma…
The classical state-space approach to optimal estimation of stochastic processes is efficient when the driving noises are generated by martingales. In particular, the weight function of the optimal linear filter, which solves a complicated operator equation in general, simplifies to the Riccati ordinary differential equation in the martingale case. This reduction lies in the foundations of the Kalman-Bucy approach to linear optimal filtering. In this paper we consider a basic Kalman-Bucy model with noises, generated by independent fractional Brownian motions, and develop a new method of asympt…
Solutions of martingale problems for Lévy-type operators and stochastic differential equations driven by Lévy processes with discontinuous coefficients
Martingales constitute a basic tool in stochastic analysis; this paper considers their application to counting processes. We use this tool to revisit a renewal theorem and its extensions for various counting processes. We first consider a renewal process as a pilot example, deriving a new semimartingale representation that differs from the standard decomposition via the stochastic intensity function. We then revisit Blackwell's renewal theorem, its refinements and extensions. Based on these observations, we extend the semimartingale representation to a general counting process, and give condit…
We investigate the properties of a discrete-time martingale $\{X_m\}_{m\in \mathbb Z_{\geq 0}}$, where all differences between adjacent random variables are limited to be not more than a constant as a promise. In this situation, it is known that the Azuma-Hoeffding inequality holds, which gives an upper bound of a probability for exceptional events. The inequality gives a simple form of the upper bound, and it has been utilized for many investigations. However, the inequality is not tight. We give an explicit expression of a tight upper bound, and we show that it and the bound obtained from th…
This paper is devoted to the asymptotic analysis of the amnesic elephant random walk (AERW) using a martingale approach. More precisely, our analysis relies on asymptotic results for multidimensional martingales with matrix normalization. In the diffusive and critical regimes, we establish the almost sure convergence and the quadratic strong law for the position of the AERW. The law of iterated logarithm is given in the critical regime. The distributional convergences of the AERW to Gaussian processes are also provided. In the superdiffusive regime, we prove the distributional convergence as w…
We investigate the asymptotic behaviour of networks of interacting non-linear Hawkes processes modeling a homogeneous population of neurons in the large population limit. In particular, we prove a functional central limit theorem for the mean spike-activity thereby characterizing the asymptotic fluctuations in terms of a stochastic Volterra integral equation. Our approach differs from previous approaches in making use of the associated resolvent in order to represent the fluctutations as Skorokhod continuous mappings of weakly converging martingales. Since the Lipschitz properties of the resol…
We introduce a new method for obtaining quantitative convergence rates for the central limit theorem (CLT) in a high dimensional setting. Using our method, we obtain several new bounds for convergence in transportation distance and entropy, and in particular: (a) We improve the best known bound, obtained by the third named author, for convergence in quadratic Wasserstein transportation distance for bounded random vectors; (b) We derive the first non-asymptotic convergence rate for the entropic CLT in arbitrary dimension, for general log-concave random vectors; (c) We give an improved bound for…
We continue the approach in Part I \cite{duchong19} to study stationary states of controlled differential equations driven by rough paths, using the framework of random dynamical systems and random attractors. Part II deals with driving paths of finite $ν$ - Hölder norms with $ν\in (\frac{1}{3},\frac{1}{2})$ so that the integrals are interpreted in the Gubinelli sense for controlled rough paths. We prove sufficient conditions for the attractor to be a singleton, thus the pathwise convergence is in both pullback and forward senses.
Let $(Z_t)_{t\geq 0}$ denote the derivative martingale of branching Brownian motion, i.e.\@ the derivative with respect to the inverse temperature of the normalized partition function at critical temperature. A well-known result by Lalley and Sellke [\textit{Ann. Probab.}, 15(3):1052--1061, 1987] says that this martingale converges almost surely to a limit $Z_\infty$, positive on the event of survival. In this paper, our concern is the fluctuations of the derivative martingale around its limit. A corollary of our results is the following convergence, confirming and strengthening a conjecture b…
