We consider a class of stochastic impulse control problems of general stochastic processes i.e. not necessarily Markovian. Under fairly general conditions we establish existence of an optimal impulse control. We also prove existence of combined optimal stochastic and impulse control of a fairly general class of diffusions with random coefficients. Unlike, in the Markovian framework, we cannot apply quasi-variational inequalities techniques. We rather derive the main results using techniques involving reflected BSDEs and the Snell envelope.
This paper studies an optimal stochastic impulse control problem in a finite horizon with a decision lag, by which we mean that after an impulse is made, a fixed number units of time has to be elapsed before the next impulse is allowed to be made. The continuity of the value function is proved. A suitable version of dynamic programming principle is established, which takes into account the dependence of state process on the elapsed time. The corresponding Hamilton-Jacobi-Bellman (HJB) equation is derived, which exhibit some special feature of the problem. The value function of this optimal imp…
We consider stochastic impulse control problems when the impulses cost functions are arbitrary. We use the dynamic programming principle and viscosity solutions approach to show that the value function is a unique viscosity solution for the associated Hamilton-Jacobi-Bellman equation (HJB) partial differential equation (PDE) of stochastic impulse control problems
This paper considers the problem of two-player zero-sum stochastic differential game with both players adopting impulse controls in finite horizon under rather weak assumptions on the cost functions ($c$ and $χ$ not decreasing in time). We use the dynamic programming principle and viscosity solutions approach to show existence and uniqueness of a solution for the Hamilton-Jacobi-Bellman-Isaacs (HJBI) partial differential equation (PDE) of the game. We prove that the upper and lower value functions coincide.
We consider a general class of nonzero-sum $N$-player stochastic games with impulse controls, where players control the underlying dynamics with discrete interventions. We adopt a verification approach and provide sufficient conditions for the Nash equilibria (NEs) of the game. We then consider the limit situation of $N \to \infty$, that is, a suitable mean-field game (MFG) with impulse controls. We show that under appropriate technical conditions, the existence of unique NE solution to the MFG, which is an $ε$-NE approximation to the $N$-player game, with $ε=O\left(\frac{1}{\sqrt{N}}\right)$.…
In this paper, a characterization of the solution of impulse control problems in terms of superharmonic functions is given. In a general Markovian framework, the value function of the impulse control problem is shown to be the minimal function in a convex set of superharmonic functions. This characterization also leads to optimal impulse control strategies and can be seen as the corresponding characterization to the description of the value function for optimal stopping problems as a smallest superharmonic majorant of the reward function. The results are illustrated with examples from differen…
We study a class of infinite horizon impulse control problems with execution delay when the dynamics of the system is described by a general adapted stochastic process. The problem is solved by means of probabilistic tools relying on the notion of Snell envelope and infinite horizon reflected backward stochastic differential equations. This allows us to establish the existence of an optimal strategy over all admissible strategies.
We study a two-player nonzero-sum stochastic differential game where one player controls the state variable via additive impulses while the other player can stop the game at any time. The main goal of this work is characterize Nash equilibria through a verification theorem, which identifies a new system of quasi-variational inequalities whose solution gives equilibrium payoffs with the correspondent strategies. Moreover, we apply the verification theorem to a game with a one-dimensional state variable, evolving as a scaled Brownian motion, and with linear payoff and costs for both players. Two…
This paper is related to nonzero-sum stochastic differential games in the Markovian framework. We show existence of a Nash equilibrium point for the game when the drift is no longer bounded and only satisfies a linear growth condition. The main tool is the notion of backward stochastic differential equations which, in our case, are multidimensional with continuous coefficient and stochastic linear growth.
We study a two-player zero-sum stochastic differential game with both players adopting impulse controls, on a finite time horizon. The Hamilton-Jacobi-Bellman-Isaacs (HJBI) partial differential equation of the game turns out to be a double-obstacle quasi-variational inequality, therefore the two obstacles are implicitly given. We prove that the upper and lower value functions coincide, indeed we show, by means of the dynamic programming principle for the stochastic differential game, that they are the unique viscosity solution to the HJBI equation, therefore proving that the game admits a valu…
In many applications, the common assumption that a driving noise process affecting a system is independent or Markovian may not be realistic, but the noise process may be assumed to be stationary. To study such problems, this paper investigates stochastic stability properties of a class of non-Markovian processes, where the existence of a stationary measure, asymptotic mean stationarity and ergodicity conditions are studied. Applications in feedback quantization and stochastic control are presented.
In this paper, we investigate the open-loop and weak closed-loop solvabilities of stochastic linear quadratic (LQ, for short) optimal control problem of Markovian regime switching system. Interestingly, these two solvabilities are equivalent. We first provide an alternative characterization of the open-loop solvability of the LQ problem using the perturbation approach. Then, we study the weak closed-loop solvability of the LQ problem of Markovian regime switching system, and establish the equivalent relationship between open-loop and weak closed-loop solvabilities. Finally, we present an examp…
In this survey we present the near-optimal stochastic control problem according to some recent tools in the literature. In particular, we focus on the approach of a discretization of the noise values instead of the canonical time-discretization. This is the so called {\it skeleton} structure. This allows to obtain an $ε$-optimal control in non-Markovian systems (the main Theorem). A simple example illustrates the technique. The importance of the approach is emphasised in a final section on open problems related to more geometrical framework and discontinuous noise.
The instantaneous state of a neural network consists of both the degree of excitation of each neuron the network is composed of and positions of impulses in communication lines between the neurons. In neurophysiological experiments, the neuronal firing moments are registered, but not the state of communication lines. But future spiking moments depend essentially on the past positions of impulses in the lines. This suggests, that the sequence of intervals between firing moments (inter-spike intervals, ISIs) in the network could be non-Markovian. In this paper, we address this question for a sim…
This article addresses a new class of fractional nonlocal neutral stochastic differential system of order 1<q<2 including non-instantaneous impulses(NIIs) and state-dependent delay(SDD) with the Poisson jumps and the Wiener process in Hilbert spaces. We examine the existence of a mild solution without assuming the compactness of the resolvent operators. The Monch fixed point theorem is used along with the Hausdorff measure of noncompactness. An example is constructed to verify the theoretical developments.
This paper considers the problem of uniqueness of the solutions to a class of Markovian backward stochastic differential equations (BSDEs) which are also connected to certain nonlinear partial differential equation (PDE) through a probabilistic representation. Assuming that there is a solution to the BSDE or to the corresponding PDE, we use the probabilistic interpretation to show the uniqueness of the solutions, and provide an example of a stochastic control application.
It is known that Markovian forward-backward stochastic differential equations provide nonlinear Feynman-Kac representation formulae for semilinear parabolic PDEs. We show that non-Markovian forward-backward stochastic differential equations provide nonlinear Feynman-Kac formulae for semilinear path-dependent PDEs. This extends the result proved in Ekren, Keller, Touzi, and Zhang [4] to the case with a possibly degenerate diffusion coefficient in the forward dynamics.
In this article, we deal with a multiple dimensional coupled Markovian BSDEs system with stochastic linear growth generators with respect to volatility processes. An existence result is provided by using approximation techniques.
We prove the convergence at an exponential rate towards the invariant probability measure for a class of solutions of stochastic differential equations with finite delay. This is done, in this non-Markovian setting, using the cluster expansion method, inspired from [4] or [14]. As a consequence, the results hold for small perturbations of ergodic diffusions.
This paper is concerned with a class of stochastic differential equations with Markovian switching. The Malliavin calculus is used to study the smoothness of the density of the solution under a Hörmander type condition. Furthermore, we obtain a Bismut type formula which is used to establish the strong Feller property.
We consider stochastic (partial) differential equations appearing as Markovian lifts of affine Volterra processes with jumps from the point of view of the generalized Feller property which was introduced in e.g.~\cite{doetei:10}. In particular we provide new existence, uniqueness and approximation results for Markovian lifts of affine rough volatility models of general jump diffusion type. We demonstrate that in this Markovian light the theory of stochastic Volterra processes becomes almost classical.
The instantaneous state of a neural network consists of both the degree of excitation of each neuron, the network is composed of, and positions of impulses in communication lines between neurons. In neurophysiological experiments, the neuronal firing moments are registered, but not the state of communication lines. But future spiking moments depend essentially on the past positions of impulses in the lines. This suggests, that the sequence of intervals between firing moments (interspike intervals, ISIs) in the network could be non-Markovian. In this paper, we address this question for a simple…
This paper explores the relationship between non-Markovian fully coupled forward-backward stochastic systems and path-dependent PDEs. The definition of classical solution for the path-dependent PDE is given within the framework of functional Itô calculus. Under mild hypotheses, we prove that the forward-backward stochastic system provides the unique classical solution to the path-dependent PDE.
We develop a theory for continuous-time non-Markovian stochastic control problems which are inherently time-inconsistent. Their distinguishing feature is that the classical Bellman optimality principle no longer holds. Our formulation is cast within the framework of a controlled non-Markovian forward stochastic differential equation, and a general objective functional setting. We adopt a game-theoretic approach to study such problems, meaning that we seek for sub-game perfect Nash equilibrium points. As a first novelty of this work, we introduce and motivate a refinement of the definition of e…
We study a stochastic SIS (susceptible-infected-susceptible) epidemic dynamics on network, under the effect of a Markovian regime-switching. We first prove the existence of a unique global positive solution, and find a positive invariant set for the system. Then, we find sufficient conditions for a.s. extinction and stochastic permanence, showing also their relation with the stationary probability distribution of the Markov chain that governs the switching and with the network topology. We provide an asymptotic lower bound for the time average of the sample-path solution under the conditions e…
We establish existence and uniqueness for a wide class of Markovian systems of backward stochastic differential equations (BSDE) with quadratic nonlinearities. This class is characterized by an abstract structural assumption on the generator, an a-priori local-boundedness property, and a locally-Hölder-continuous terminal condition. We present easily verifiable sufficient conditions for these assumptions and treat several applications, including stochastic equilibria in incomplete financial markets, stochastic differential games, and martingales on Riemannian manifolds.
Nonzero-sum stochastic differential games with impulse controls: a verification theorem with applications
London School of Economics and Political Science（伦敦政治经济学院）；University Paris Dauphine（巴黎第九大学）；Finance For Energy Market Research Centre (FiME)（能源金融市场研究中心）；University of California, Berkeley（加州大学伯克利分校）；University of Padova（帕多瓦大学）
A feature often observed in epidemiological networks is significant heterogeneity in degree. A popular modelling approach to this has been to consider large populations with highly heterogeneous discrete contact rates. This paper defines an individual-level non-Markovian stochastic process that converges on standard ODE models of such populations in the appropriate asymptotic limit. A generalised Sellke construction is derived for this model, and this is then used to consider final outcomes in the case where heterogeneity follows a truncated Zipf distribution.
Order-preserving couplings are elegant tools for obtaining robust estimates of the time-dependent and stationary distributions of Markov processes that are too complex to be analyzed exactly. The starting point of this paper is to study stochastic relations, which may be viewed as natural generalizations of stochastic orders. This generalization is motivated by the observation that for the stochastic ordering of two Markov processes, it suffices that the generators of the processes preserve some, not necessarily reflexive or transitive, subrelation of the order relation. The main contributions…
One of the fundamental assumptions in stochastic control of continuous time processes is that the dynamics of the underlying (diffusion) process is known. This is, however, usually obviously not fulfilled in practice. On the other hand, over the last decades, a rich theory for nonparametric estimation of the drift (and volatility) for continuous time processes has been developed. The aim of this paper is bringing together techniques from stochastic control with methods from statistics for stochastic processes to find a way to both learn the dynamics of the underlying process and control in a r…
A class of stochastic delay equations in Banach space $E$ driven by cylindrical Wiener process is studied. We investigate two concepts of solutions: weak and generalised strong, and give conditions under which they are equivalent. We present an evolution equation approach in a Banach space $\Ep{p}:=E\times L^p(-1,0;E)$ proving that the solutions can be reformulated as $\Ep{p}$-valued Markov processes. Based on the Markovian representation we prove the existence and continuity of the solutions. The results are applied to stochastic delay partial differential equations with an application to neu…
An explicit Milstein-type scheme for stochastic differential equation with Markovian switching is derived and its strong convergence in $\mathcal{L}^2$-sense is established without using Itô-Taylor expansion formula. Rate of strong convergence is shown to be equal to $1.0$ under the assumptions that coefficients satisfy mild regularity conditions. More precisely, coefficients are assumed to be only once differentiable which are more relaxed conditions than those made in existing literature.
We propose a new tamed Milstein-type scheme for stochastic differential equation with Markovian switching when drift coefficient is assumed to grow super-linearly. The strong rate of convergence is shown to be equal to $1.0$ under mild regularity (e.g. once differentiability) requirements on drift and diffusion coefficients. Novel techniques are developed to tackle two-fold difficulties arising due to jumps of the Markov chain and the reduction of regularity requirements on the coefficients.
Two discretizations of a class of locally Lipschitz Markovian backward stochastic differential equations (BSDEs) are studied. The first is the classical Euler scheme which approximates a projection of the processes Z, and the second a novel scheme based on Malliavin weights which approximates the mariginals of the process Z directly. Extending the representation theorem of Ma and Zhang leads to advanced a priori estimates and stability results for this class of BSDEs. These estimates are then used to obtain competitive convergence rates for both schemes with respect to the number of points in …
The aim of this paper is to study the dynamical behavior of non-autonomous stochastic lattice systems with Markovian switching. We first show existence of an evolution system of measures of the stochastic system. We then study the pullback (or forward) asymptotic stability in distribution of the evolution system of measures. We finally prove that any limit point of a tight sequence of an evolution system of measures of the stochastic lattice systems must be an evolution system of measures of the corresponding limiting system as the intensity of noise converges zero. In particular, when the coe…
The existence and uniqueness of the numerical invariant measure of the backward Euler-Maruyama method for stochastic differential equations with Markovian switching is yielded, and it is revealed that the numerical invariant measure converges to the underlying invariant measure in the Wasserstein metric. Under the polynomial growth condition of drift term the convergence rate is estimated. The global Lipschitz condition on the drift coefficients required by Bao et al., 2016 and Yuan et al., 2005 is released. Several examples and numerical experiments are given to verify our theory.
We introduce Markovian cocycle perturbations of the groups of transformations associated with the classical and quantum stochastic processes with stationary increments, which are characterized by a localization of the perturbation to the algebra of events of the past. It is namely the definition one needs because the Markovian perturbations of the Kolmogorov flows associated with the classical and quantum noises result in the perturbed group of transformations which can be decomposed in the sum of a part associated with deterministic stochastic processes lying in the past and a part associated…
We consider rough stochastic volatility models where the variance process satisfies a stochastic Volterra equation with the fractional kernel, as in the rough Bergomi and the rough Heston model. In particular, the variance process is therefore not a Markov process or semimartingale, and has quite low Hölder-regularity. In practice, simulating such rough processes thus often results in high computational cost. To remedy this, we study approximations of stochastic Volterra equations using an $N$-dimensional diffusion process defined as solution to a system of ordinary stochastic differential equ…
We study two-dimensional stochastic differential equations (SDEs) of McKean--Vlasov type in which the conditional distribution of the second component of the solution given the first enters the equation for the first component of the solution. Such SDEs arise when one tries to invert the Markovian projection developed by Gyöngy (1986), typically to produce an Itô process with the fixed-time marginal distributions of a given one-dimensional diffusion but richer dynamical features. We prove the strong existence of stationary solutions for these SDEs, as well as their strong uniqueness in an impo…
Stochastic approximation is a framework unifying many random iterative algorithms occurring in a diverse range of applications. The stability of the process is often difficult to verify in practical applications and the process may even be unstable without additional stabilisation techniques. We study a stochastic approximation procedure with expanding projections similar to Andradóttir [Oper. Res. 43 (1995) 1037-1048]. We focus on Markovian noise and show the stability and convergence under general conditions. Our framework also incorporates the possibility to use a random step size sequence,…
We consider the linear stochastic recursion $x_{i+1} = a_{i}x_{i}+b_{i}$ where the multipliers $a_i$ are random and have Markovian dependence given by the exponential of a standard Brownian motion and $b_{i}$ are i.i.d. positive random noise independent of $a_{i}$. Using large deviations theory we study the growth rates (Lyapunov exponents) of the positive integer moments $λ_q = \lim_{n\to \infty} \frac{1}{n} \log\mathbb{E}[(x_n)^q]$ with $q\in \mathbb{Z}_+$. We show that the Lyapunov exponents $λ_q$ exist, under appropriate scaling of the model parameters, and have non-analytic behavior manif…
