Unique existence of analytically strong solutions to stochastic partial differential equations (SPDE) with drift given by the subdifferential of a quasi-convex function and with general multiplicative noise is proven. The proof applies a genuinely new method of weighted Galerkin approximations based on the "distance" defined by the quasi-convex function. Spatial regularization of the initial condition analogous to the deterministic case is obtained. The results yield a unified framework which is applied to stochastic generalized porous media equations, stochastic generalized reaction diffusion equations and stochastic generalized degenerated p-Laplace equations. In particular, higher regularity for solutions of such SPDE is obtained.
Under general conditions we show that the solution of a stochastic parabolic partial differential equation of the form \[ \partial_t u = \mathrm{div} (A \nabla u) + f(t,x, u) + g_i (t,x,u) \dot{w}^i_t \] is almost surely Hölder continuous in both space and time variables.
Using a multiplicative structure (for example that of a Banach algebra) and a partial order we construct a weak version of a Banach space valued stochastic integral with respect to square integrable martingales.
Universität Bielefeld（比勒费尔德大学）；Max Planck Institute for Mathematics in the Sciences（马克斯·普朗克数学研究所）；Institute of Mathematics of NAS of Ukraine（乌克兰国家科学院数学研究所）
We propose new limiting dynamics for stochastic gradient descent in the small learning rate regime called stochastic modified flows. These SDEs are driven by a cylindrical Brownian motion and improve the so-called stochastic modified equations by having regular diffusion coefficients and by matching the multi-point statistics. As a second contribution, we introduce distribution dependent stochastic modified flows which we prove to describe the fluctuating limiting dynamics of stochastic gradient descent in the small learning rate - infinite width scaling regime.
We prove the existence of random dynamical systems and random attractors for a large class of locally monotone stochastic partial differential equations perturbed by additive Lévy noise. The main result is applicable to various types of SPDE such as stochastic Burgers type equations, stochastic 2D Navier-Stokes equations, the stochastic 3D Leray-$α$ model, stochastic power law fluids, the stochastic Ladyzhenskaya model, stochastic Cahn-Hilliard type equations, stochastic Kuramoto-Sivashinsky type equations, stochastic porous media equations and stochastic $p$-Laplace equations.
Random invariant manifolds are geometric objects useful for understanding complex dynamics under stochastic influences. Under a nonuniform hyperbolicity or a nonuniform exponential dichotomy condition, the existence of random pseudo-stable and pseudo-unstable manifolds for a class of \emph{random} partial differential equations and \emph{stochastic} partial differential equations is shown. Unlike the invariant manifold theory for stochastic \emph{ordinary} differential equations, random norms are not used. The result is then applied to a nonlinear stochastic partial differential equation with …
In this paper we show that solutions of stochastic partial differential equations driven by Brownian motion can be approximated by stochastic partial differential equations forced by pure jump noise/random kicks. Applications to stochastic Burgers equations are discussed.
We study in this paper a weak approximation to stochastic variance reduced gradient Langevin dynamics by stochastic delay differential equations in Wasserstein-1 distance, and obtain a uniform error bound. Our approach is via a refined Lindeberg principle and Malliavin calculus.
In the machine learning literature stochastic gradient descent has recently been widely discussed for its purported implicit regularization properties. Much of the theory, that attempts to clarify the role of noise in stochastic gradient algorithms, has widely approximated stochastic gradient descent by a stochastic differential equation with Gaussian noise. We provide a novel rigorous theoretical justification for this practice that showcases how the Gaussianity of the noise arises naturally.
In this paper we use methods from Stochastic Analysis to establish Li-Yau type estimates for positive solutions of the heat equation. In particular, we want to emphasize that Stochastic Analysis provides natural tools to derive local estimates in the sense that the gradient bound at given point depends only on universal constants and the geometry of the Riemannian manifold locally about this point.
This paper is devoted to proving the strong averaging principle for slow-fast stochastic partial differential equations with locally monotone coefficients, where the slow component is a stochastic partial differential equations with locally monotone coefficients and the fast component is a stochastic partial differential equations (SPDEs) with strongly monotone coefficients. The result is applicable to a large class of examples, such as the stochastic porous medium equation, the stochastic $p$-Laplace equation, the stochastic Burgers type equation and the stochastic 2D Navier-Stokes equation, …
We trace the evolution of the theory of stochastic partial differential equations from the foundation to its development, until the recent solution of long-standing problems on well-posedness of the KPZ equation and the stochastic quantization in dimension three.
In this paper, we establish a central limit theorem (CLT) and the moderate deviation principles (MDP) for a class of semilinear stochastic partial differential equations driven by multiplicative noise on a bounded domain. The main results can be applied to stochastic partial differential equations of various types such as the stochastic Burgers equation and the reaction-diffusion equations perturbed by space-time white noise.
In this article, we study elliptic stochastic partial differential equations with two reflect- ing walls h1 and h2, driven by multiplicative noise. The existence and uniqueness of the solutions are established.
We prove a priori estimates in $L_\infty$ for a class of quasilinear stochastic partial differential equations. The estimates are obtained independently of the ellipticity constant $\varepsilon$ and thus imply analogous estimates for degenerate quasilinear stochastic partial differential equations, such as the stochastic porous medium equation.
In this paper we investigate classical solution of a semi-linear system of backward stochastic integral partial differential equations driven by a Brownian motion and a Poisson point process. By proving an Itô-Wentzell formula for jump diffusions as well as an abstract result of stochastic evolution equations, we obtain the stochastic integral partial differential equation for the inverse of the stochastic flow generated by a stochastic differential equation driven by a Brownian motion and a Poisson point process. By composing the random field generated by the solution of a backward stochastic…
In this paper parabolic random partial differential equations and parabolic stochastic partial differential equations driven by a Wiener process are considered. A deterministic, tensorized evolution equation for the second moment and the covariance of the solutions of the parabolic stochastic partial differential equations is derived. Well-posedness of a space-time weak variational formulation of this tensorized equation is established.
We prove, under mild conditions, that the stochastic gradient Langevin dynamics converges to a limiting law as time tends to infinity, even in the case where the driving data sequence is dependent.
In this paper, we establish the Freidlin-Wentzell's large deviations for quasilinear parabolic stochastic partial differential equations with multiplicative noise, which are neither monotone nor locally monotone. The proof is based on the weak convergence approach.
In this paper, we deal with a class of backward doubly stochastic differential equations (BDSDEs, in short) involving subdifferential operator of a convex function and driven by Teugels martingales associated with a Lévy process. We show the existence and uniqueness result by means of Yosida approximation. As an application, we give the existence of stochastic viscosity solution for a class of multivalued stochastic partial differential-integral equations (MSPIDEs, in short).
Stochastic gradient descent is an optimisation method that combines classical gradient descent with random subsampling within the target functional. In this work, we introduce the stochastic gradient process as a continuous-time representation of stochastic gradient descent. The stochastic gradient process is a dynamical system that is coupled with a continuous-time Markov process living on a finite state space. The dynamical system -- a gradient flow -- represents the gradient descent part, the process on the finite state space represents the random subsampling. Processes of this type are, fo…
In stochastic optimization, a common tool to deal sequentially with large sample is to consider the well-known stochastic gradient algorithm. Nevertheless, since the stepsequence is the same for each direction, this can lead to bad results in practice in case of ill-conditionned problem. To overcome this, adaptive gradient algorithms such that Adagrad or Stochastic Newton algorithms should be prefered. This paper is devoted to the non asymptotic analyis of these adaptive gradient algorithms for strongly convex objective. All the theoretical results will be adapted to linear regression and regu…
Fractional Backward Stochastic Partial Differential Equations with Applications to Stochastic Optimal Control of Partially Observed Systems driven by Lévy Processes
Non-asymptotic convergence analysis of the stochastic gradient Hamiltonian Monte Carlo algorithm with discontinuous stochastic gradient with applications to training of ReLU neural networks
School of Mathematics, Renmin University of China, Beijing, China（中国人民大学数学学院）；Division of Mathematical Sciences, Nanyang Technological University, 21 Nanyang Link, 637371 Singapore（南洋理工大学数学科学学院）；Financial Technology Thrust, Society Hub, The Hong Kong University of Science and Technology (Guangzhou), Guangzhou, China（香港科技大学（广州）金融科技中心）
We establish a Berry--Esseen bound for general multivariate nonlinear statistics by developing a new multivariate-type randomized concentration inequality. The bound is the best possible for many known statistics. As applications, Berry--Esseen bounds for M-estimators and averaged stochastic gradient descent algorithms are obtained.
A large deviation principle is derived for stochastic partial differential equations with slow-fast components. The result shows that the rate function is exactly that of the averaged equation plus the fluctuating deviation which is a stochastic partial differential equation with small Gaussian perturbation. This also confirms the effectiveness of the approximation of the averaged equation plus the fluctuating deviation to the slow-fast stochastic partial differential equations.
We propose an online parametric estimation method of stochastic differential equations with discrete observations and misspecified modelling based on online gradient descent. Our study provides uniform upper bounds for the risks of the estimators over a family of stochastic differential equations. The derivation of the bounds involves three underlying theoretical results: the analysis of the stochastic mirror descent algorithm based on dependent and biased subgradients, the simultaneous exponential ergodicity of classes of diffusion processes, and the proposal of loss functions whose approxima…
We explore Ito stochastic differential equations where the drift term possibly depends on the infinite past. Assuming the existence of a Lyapunov function, we prove the existence of a stationary solution assuming only minimal continuity of the coefficients. Uniqueness of the stationary solution is proven if the dependence on the past decays sufficiently fast. The results of this paper are then applied to stochastically forced dissipative partial differential equations such as the stochastic Navier-Stokes equation and stochastic Ginsburg-Landau equation.
