We prove concentration inequalities for functions of independent random variables {under} sub-gaussian and sub-exponential conditions. The utility of the inequalities is demonstrated by an extension of the now classical method of Rademacher complexities to Lipschitz function classes and unbounded sub-exponential distribution.

2021-06-23cs.LGmath.PRstat.ML

Concentration inequalities and optimal number of layers for stochastic deep neural networks

University of Pennsylvania（宾夕法尼亚大学）；Universität Leipzig（莱比锡大学）；Max Planck Institute for Mathematics in the Sciences（马克斯·普朗克数学科学研究所）；Duke University（杜克大学）

We state concentration inequalities for the output of the hidden layers of a stochastic deep neural network (SDNN), as well as for the output of the whole SDNN. These results allow us to introduce an expected classifier (EC), and to give probabilistic upper bound for the classification error of the EC. We also state the optimal number of layers for the SDNN via an optimal stopping procedure. We apply our analysis to a stochastic version of a feedforward neural network with ReLU activation function.

2023-03-20cs.LGstat.ML

A Short Note on Concentration Inequalities for Random Vectors with SubGaussian Norm

University of California, Berkeley（加州大学伯克利分校）；Microsoft Research, India（微软研究院印度）；Duke University（杜克大学）；University of Washington, Seattle（华盛顿大学西雅图分校）

In this note, we derive concentration inequalities for random vectors with subGaussian norm (a generalization of both subGaussian random vectors and norm bounded random vectors), which are tight up to logarithmic factors.

2019-02-11cs.LGmath.PRstat.ML

Concentration Inequalities for Bounded Random Vectors

Louisiana State University（路易斯安那州立大学）；Southern University and A&M College（南方大学及农工学院）

We derive simple concentration inequalities for bounded random vectors, which generalize Hoeffding's inequalities for bounded scalar random variables. As applications, we apply the general results to multinomial and Dirichlet distributions to obtain multivariate concentration inequalities.

2013-08-30cs.LGmath.PRmath.ST

Introduction to Concentration Inequalities

IIIT Hyderabad（海得拉巴国际信息技术学院）；IIT Roorkee（鲁尔基印度理工学院）

In this report, we aim to exemplify concentration inequalities and provide easy to understand proofs for it. Our focus is on the inequalities which are helpful in the design and analysis of machine learning algorithms.

2019-10-04cs.LGmath.PRmath.ST

Some compact notations for concentration inequalities and user-friendly results

Princeton University（普林斯顿大学）

This paper presents compact notations for concentration inequalities and convenient results to streamline probabilistic analysis. The new expressions describe the typical sizes and tails of random variables, allowing for simple operations without heavy use of inessential constants. They bridge classical asymptotic notations and modern non-asymptotic tail bounds together. Examples of different kinds demonstrate their efficacy.

2020-04-26cs.ITcs.LGmath.IT

更多相关论文收起更多论文

Transportation-Inequalities, Lyapunov Stability and Sampling for Dynamical Systems on Continuous State Space

Duke University（杜克大学）

We study the concentration phenomenon for discrete-time random dynamical systems with an unbounded state space. We develop a heuristic approach towards obtaining exponential concentration inequalities for dynamical systems using an entirely functional analytic framework. We also show that existence of exponential-type Lyapunov function, compared to the purely deterministic setting, not only implies stability but also exponential concentration inequalities for sampling from the stationary distribution, via \emph{transport-entropy inequality} (T-E). These results have significant impact in \emph…

2022-12-07cs.ITcs.LGcs.SY

统计推断中的集中不等式

Concentration Inequalities for Statistical Inference

Center for Statistical Sciences, Peking University（北京大学统计科学中心）；Department of Mathematics, Faculty of Science and Technology, University of Macau（澳门大学科技学院数学系）

本文综述了多场景下数理统计非渐近分析所用的集中不等式，补充新结果，给出高维线性与Poisson回归场景的相关结论，旨在明确已知常数的不等式并以更优常数改进现有界。

2025-02-21cs.LGmath.PRmath.ST

向量值自归一化集中不等式超越子高斯性

Vector-valued self-normalized concentration inequalities beyond sub-Gaussianity

Department of Statistics & Data Science（统计与数据科学系）；Machine Learning Department（机器学习系）；Carnegie Mellon University（卡内基梅隆大学）

本文研究了超越子高斯性的向量值自归一化过程的集中不等式，提出适用于轻尾分布（如Bennett或Bernstein界）的界，并在在线线性回归中应用。

2026-05-14cs.LGmath.STstat.ML

留一法交叉验证的集中不等式

Concentration inequalities for leave-one-out cross validation

Uppsala University（乌普萨拉大学）

本文证明估计器稳定性足以保证留一法交叉验证的可靠性，通过一般框架下的集中界，并利用对数Sobolev不等式，涵盖线性回归、核密度估计等例子。

2026-10-14cs.LGmath.STstat.ML

A Likelihood Ratio Approach for Probabilistic Inequalities

Louisiana State University（路易斯安那州立大学）；Southern University and A&M College（南方大学及农工学院）

We propose a new approach for deriving probabilistic inequalities based on bounding likelihood ratios. We demonstrate that this approach is more general and powerful than the classical method frequently used for deriving concentration inequalities such as Chernoff bounds. We discover that the proposed approach is inherently related to statistical concepts such as monotone likelihood ratio, maximum likelihood, and the method of moments for parameter estimation. A connection between the proposed approach and the large deviation theory is also established. We show that, without using moment gener…

2013-08-18cs.LGmath.PRmath.ST