We prove an extension of McDiarmid's inequality for metric spaces with unbounded diameter. To this end, we introduce the notion of the {\em subgaussian diameter}, which is a distribution-dependent refinement of the metric diameter. Our technique provides an alternative approach to that of Kutin and Niyogi's method of weakly difference-bounded functions, and yields nontrivial, dimension-free results in some interesting cases where the former does not. As an application, we give apparently the first generalization bound in the algorithmic stability setting that holds for unbounded loss functions. We furthermore extend our concentration inequality to strongly mixing processes.
We prove semi-empirical concentration inequalities for random variables which are given as possibly nonlinear functions of independent random variables. These inequalities describe concentration of random variable in terms of the data/distribution-dependent Efron-Stein (ES) estimate of its variance and they do not require any additional assumptions on the moments. In particular, this allows us to state semi-empirical Bernstein type inequalities for general functions of unbounded random variables, which gives user-friendly concentration bounds for cases where related methods (e.g. bounded diffe…
We consider the simple random walk in i.i.d. nonnegative potentials on the multidimensional cubic lattice. Our goal is to investigate the cost paid by the simple random walk for traveling from the origin to a remote location in a landscape of potentials. In particular, we obtain concentration inequalities for the travel cost in unbounded nonnegative potentials.
We present new algorithms for online convex optimization over unbounded domains that obtain parameter-free regret in high-probability given access only to potentially heavy-tailed subgradient estimates. Previous work in unbounded domains considers only in-expectation results for sub-exponential subgradients. Unlike in the bounded domain case, we cannot rely on straight-forward martingale concentration due to exponentially large iterates produced by the algorithm. We develop new regularization techniques to overcome these problems. Overall, with probability at most $δ$, for all comparators $\ma…
We study the free energy and its relevant quantity for the directed polymer in random environment. The polymer is allowed to make unbounded jumps and the environment is given by the Bernoulli variables. We first establish the concentration of the ground state energy of polymer at zero temperature. Secondly, we also prove the same property of the free energy at finite temperature. In the proof, we use the fact that the maximum jump of any polymer nearly minimizing energy is not too large with high probability. This is an interesting property itself from the first passage percolation viewpoint.
Submodular and fractionally subadditive (or equivalently XOS) functions play a fundamental role in combinatorial optimization, algorithmic game theory and machine learning. Motivated by learnability of these classes of functions from random examples, we consider the question of how well such functions can be approximated by low-degree polynomials in $\ell_2$ norm over the uniform distribution. This question is equivalent to understanding of the concentration of Fourier weight on low-degree coefficients, a central concept in Fourier analysis. We show that 1. For any submodular function $f:\{0,1…
We prove concentration inequalities for functions of independent random variables {under} sub-gaussian and sub-exponential conditions. The utility of the inequalities is demonstrated by an extension of the now classical method of Rademacher complexities to Lipschitz function classes and unbounded sub-exponential distribution.
We obtain optimal Gaussian concentration bounds (GCBs) for stochastic chains of unbounded memory (SCUMs) on countable alphabets. These stochastic processes are also known as "chains with complete connections" or "$g$-measures". We consider two different conditions on the kernel: (1) when the sum of its oscillations is less than one, or (2) when the sum of its variations is finite, i.e., belongs to $\ell^1(\mathbb{N})$. We also obtain explicit constants as functions of the parameters of the model. The proof is based on maximal coupling. Our conditions are optimal in the sense that we exhibit ex…
This paper is devoted to uniform versions of the Hanson-Wright inequality for a random vector $X \in \mathbb{R}^n$ with independent subgaussian components. The core technique of the paper is based on the entropy method combined with truncations of both gradients of functions of interest and of the coordinates of $X$ itself. Our results recover, in particular, the classic uniform bound of Talagrand (1996) for Rademacher chaoses and the more recent uniform result of Adamczak (2015), which holds under certain rather strong assumptions on the distribution of $X$. We provide several applications of…
Sinkhorn Linearization and the Spectral Proxy: Unifying the Statistical and Algorithmic Theory of Feature-Parameterized Inverse Optimal Transport via a Single Spectral Sandwich
School of Medicine, Nankai University（南开大学医学院）；College of Artificial Intelligence, Nankai University（南开大学人工智能学院）
Understanding Robust Machine Learning for Nonparametric Regression with Heavy-Tailed Noise
Department of Mathematics and Statistics, State University of New York at Albany（纽约州立大学阿尔巴尼分校数学与统计学系）；Department of Mathematics, University of Tennessee, Knoxville（田纳西大学基洛纳分校数学系）
Using the method of transportation-information inequality introduced in \cite{GLWY}, we establish Bernstein type's concentration inequalities for empirical means $\frac 1t \int_0^t g(X_s)ds$ where $g$ is a unbounded observable of the symmetric Markov process $(X_t)$. Three approaches are proposed : functional inequalities approach ; Lyapunov function method ; and an approach through the Lipschitzian norm of the solution to the Poisson equation. Several applications and examples are studied.
We study Talagrand concentration and Poincaré type inequalities for unbounded pure jump Markov processes. In particular we focus on processes with degenerate jumps that depend on the past of the whole system, based on the model introduced by Galves and Löcherbach in \cite{G-L}, in order to describe the activity of a biological neural network. As a result we obtain exponential rates of convergence to equilibrium.
The concentration inequality approach for normal approximation by Stein's method is generalized to the multivariate setting. We use this approach to prove a non-smooth function distance for multivariate normal approximation for standardized sums of $k$-dimensional independent random vectors $W=\sum_{i=1}^n X_i$ with an error bound of order $k^{1/2}γ$ where $γ=\sum_{i=1}^n E|X_i|^3$. For sums of locally dependent (unbounded) random vectors, we obtain a fourth moment bound which is typically of order $O_k(1/\sqrt{n})$, as well as a third moment bound which is typically of order $O_k(\log n/\sqrt…
We prove regenerative properties for the linear Hawkes process under minimal assumptions on the transfer function, which may have unbounded support. These results are applicable to sliding window statistical estimators. We exploit independence in the Poisson cluster point process decomposition, and the regeneration times are not stopping times for the Hawkes process. The regeneration time is interpreted as the renewal time at zero of a M/G/infinity queue, which yields a formula for its Laplace transform. When the transfer function admits some exponential moments, we stochastically dominate the…
