We derive central limit theorems for the Wasserstein distance between the empirical distributions of Gaussian samples. The cases are distinguished whether the underlying laws are the same or different. Results are based on the (quadratic) Frechet differentiability of the Wasserstein distance in the Gaussian case. Extensions to elliptically symmetric distributions are discussed as well as several applications such as bootstrap and statistical testing.
This article is dedicated to the estimation of Wasserstein distances and Wasserstein costs between two distinct continuous distributions $F$ and $G$ on $\mathbb R$. The estimator is based on the order statistics of (possibly dependent) samples of $F$ resp. $G$. We prove the consistency and the asymptotic normality of our estimators. \begin{it}Keywords:\end{it} Central Limit Theorems- Generelized Wasserstein distances- Empirical processes- Strong approximation- Dependent samples.
The Wasserstein distance has been an attractive tool in many fields. But due to its high computational complexity and the phenomenon of the curse of dimensionality in empirical estimation, various extensions of the Wasserstein distance have been proposed to overcome the shortcomings such as the Sliced Wasserstein distance. It enjoys a low computational cost and dimension-free sample complexity, but there are few distributional limit results of it. In this paper, we focus on Sliced 1-Wasserstein distance and its variant max-Sliced 1-Wasserstein distance. We utilize the central limit theorem in …
We develop a general framework for statistical inference with the 1-Wasserstein distance. Recently, the Wasserstein distance has attracted considerable attention and has been widely applied to various machine learning tasks because of its excellent properties. However, hypothesis tests and a confidence analysis for the Wasserstein distance have not been established in a general multivariate setting. This is because the limit distribution of the empirical distribution with the Wasserstein distance is unavailable without strong restriction. To address this problem, in this study, we develop a no…
Wasserstein barycenters and variance-like criteria based on the Wasserstein distance are used in many problems to analyze the homogeneity of collections of distributions and structural relationships between the observations. We propose the estimation of the quantiles of the empirical process of Wasserstein's variation using a bootstrap procedure. We then use these results for statistical inference on a distribution registration model for general deformation functions. The tests are based on the variance of the distributions with respect to their Wasserstein's barycenters for which we prove cen…
We extend the celebrated Stone's theorem to the framework of distributional regression. More precisely, we prove that weighted empirical distribution with local probability weights satisfying the conditions of Stone's theorem provide universally consistent estimates of the conditional distributions, where the error is measured by the Wasserstein distance of order p $\ge$ 1. Furthermore, for p = 1, we determine the minimax rates of convergence on specific classes of distributions. We finally provide some applications of these results, including the estimation of conditional tail expectation or …
The Wasserstein distance is a metric on a space of probability measures that has seen a surge of applications in statistics, machine learning, and applied mathematics. However, statistical aspects of Wasserstein distances are bottlenecked by the curse of dimensionality, whereby the number of data points needed to accurately estimate them grows exponentially with dimension. Gaussian smoothing was recently introduced as a means to alleviate the curse of dimensionality, giving rise to a parametric convergence rate in any dimension, while preserving the Wasserstein metric and topological structure…
The Wasserstein distance is an attractive tool for data analysis but statistical inference is hindered by the lack of distributional limits. To overcome this obstacle, for probability measures supported on finitely many points, we derive the asymptotic distribution of empirical Wasserstein distances as the optimal value of a linear program with random objective function. This facilitates statistical inference (e.g. confidence intervals for sample based Wasserstein distances) in large generality. Our proof is based on directional Hadamard differentiability. Failure of the classical bootstrap an…
This paper is focused on the statistical analysis of probability measures $ν_{1},\ldots,ν_{n}$ on $\mathbb{R}$ that can be viewed as independent realizations of an underlying stochastic process. We consider the situation of practical importance where the random measures $ν_{i}$ are absolutely continuous with densities $f_{i}$ that are not directly observable. In this case, instead of the densities, we have access to datasets of real random variables $(X_{i,j})_{1 \leq i \leq n; \; 1 \leq j \leq p_{i} }$ organized in the form of $n$ experimental units, such that $X_{i,1},\ldots,X_{i,p_{i}}$ are…
We give a new proof of the classical Central Limit Theorem, in the Mallows ($L^r$-Wasserstein) distance. Our proof is elementary in the sense that it does not require complex analysis, but rather makes use of a simple subadditive inequality related to this metric. The key is to analyse the case where equality holds. We provide some results concerning rates of convergence. We also consider convergence to stable distributions, and obtain a bound on the rate of such convergence.
University of Chicago, Booth School of Business（芝加哥大学布斯商学院）
We study the minimax optimal rate for estimating the Wasserstein-$1$ metric between two unknown probability measures based on $n$ i.i.d. empirical samples from them. We show that estimating the Wasserstein metric itself between probability measures, is not significantly easier than estimating the probability measures under the Wasserstein metric. We prove that the minimax optimal rates for these two problems are multiplicatively equivalent, up to a $\log \log (n)/\log (n)$ factor.
In this work we introduce the concept of Bures-Wasserstein barycenter $Q_*$, that is essentially a Fréchet mean of some distribution $\mathbb{P}$ supported on a subspace of positive semi-definite Hermitian operators $\mathbb{H}_{+}(d)$. We allow a barycenter to be restricted to some affine subspace of $\mathbb{H}_{+}(d)$ and provide conditions ensuring its existence and uniqueness. We also investigate convergence and concentration properties of an empirical counterpart of $Q_*$ in both Frobenius norm and Bures-Wasserstein distance, and explain, how obtained results are connected to optimal tra…
We consider a data-driven robust hypothesis test where the optimal test will minimize the worst-case performance regarding distributions that are close to the empirical distributions with respect to the Wasserstein distance. This leads to a new non-parametric hypothesis testing framework based on distributionally robust optimization, which is more robust when there are limited samples for one or both hypotheses. Such a scenario often arises from applications such as health care, online change-point detection, and anomaly detection. We study the computational and statistical properties of the p…
Sliced Wasserstein distances preserve properties of classic Wasserstein distances while being more scalable for computation and estimation in high dimensions. The goal of this work is to quantify this scalability from three key aspects: (i) empirical convergence rates; (ii) robustness to data contamination; and (iii) efficient computational methods. For empirical convergence, we derive fast rates with explicit dependence of constants on dimension, subject to log-concavity of the population distributions. For robustness, we characterize minimax optimal, dimension-free robust estimation risks, a…
Goodness-of-fit tests based on the empirical Wasserstein distance are proposed for simple and composite null hypotheses involving general multivariate distributions. For group families, the procedure is to be implemented after preliminary reduction of the data via invariance.This property allows for calculation of exact critical values and p-values at finite sample sizes. Applications include testing for location--scale families and testing for families arising from affine transformations, such as elliptical distributions with given standard radial density and unspecified location vector and s…
We provide a short proof that the Wasserstein distance between the empirical measure of a n-sample and the estimated measure is of order n^-(1/d), if the measure has a lower and upper bounded density on the d-dimensional flat torus.
We provide some non asymptotic bounds, with explicit constants, that measure the rate of convergence, in expected Wasserstein distance, of the empirical measure associated to an i.i.d. $N$-sample of a given probability distribution on $\mathbb{R}^d$.
We study the Wasserstein distance $W_2$ for Gaussian samples. We establish the exact rate of convergence $\sqrt{\log\log n/n}$ of the expected value of the $W_2$ distance between the empirical and true $c.d.f.$'s for the normal distribution. We also show that the rate of weak convergence is unexpectedly $1/\sqrt{n}$ in the case of two correlated Gaussian samples.
We consider a sequence of identically independently distributed random samples from an absolutely continuous probability measure in one dimension with unbounded density. We establish a new rate of convergence of the $\infty-$Wasserstein distance between the empirical measure of the samples and the true distribution, which extends the previous convergence result by Trilllos and Slepčev to the case that the true distribution has an unbounded density.
Minimum distance estimation (MDE) gained recent attention as a formulation of (implicit) generative modeling. It considers minimizing, over model parameters, a statistical distance between the empirical data distribution and the model. This formulation lends itself well to theoretical analysis, but typical results are hindered by the curse of dimensionality. To overcome this and devise a scalable finite-sample statistical MDE theory, we adopt the framework of smooth 1-Wasserstein distance (SWD) $\mathsf{W}_1^{(σ)}$. The SWD was recently shown to preserve the metric and topological structure of…
The Wasserstein distance between two probability measures on a metric space is a measure of closeness with applications in statistics, probability, and machine learning. In this work, we consider the fundamental question of how quickly the empirical measure obtained from $n$ independent samples from $μ$ approaches $μ$ in the Wasserstein distance of any order. We prove sharp asymptotic and finite-sample results for this rate of convergence for general measures on general compact metric spaces. Our finite-sample results show the existence of multi-scale behavior, where measures can exhibit radic…
We provide upper bounds of the expected Wasserstein distance between a probability measure and its empirical version, generalizing recent results for finite dimensional Euclidean spaces and bounded functional spaces. Such a generalization can cover Euclidean spaces with large dimensionality, with the optimal dependence on the dimensionality. Our method also covers the important case of Gaussian processes in separable Hilbert spaces, with rate-optimal upper bounds for functional data distributions whose coordinates decay geometrically or polynomially. Moreover, our bounds of the expected value …
Let $μ_N$ be the empirical measure associated to a $N$-sample of a given probability distribution $μ$ on $\mathbb{R}^d$. We are interested in the rate of convergence of $μ_N$ to $μ$, when measured in the Wasserstein distance of order $p>0$. We provide some satisfying non-asymptotic $L^p$-bounds and concentration inequalities, for any values of $p>0$ and $d\geq 1$. We extend also the non asymptotic $L^p$-bounds to stationary $ρ$-mixing sequences, Markov chains, and to some interacting particle systems.
A number of researchers have independently introduced topologies on the set of laws of stochastic processes that extend the usual weak topology. Depending on the respective scientific background this was motivated by applications and connections to various areas (e.g. Plug-Pichler - stochastic programming, Hellwig - game theory, Aldous - stability of optimal stopping, Hoover-Keisler - model theory). Remarkably, all these seemingly independent approaches define the same adapted weak topology in finite discrete time. Our first main result is to construct an adapted variant of the empirical measu…
We consider a Markov chain on $\mathbb{R}^d$ with invariant measure $μ$. We are interested in the rate of convergence of the empirical measures towards the invariant measure with respect to various dual distances, including in particular the $1$-Wasserstein distance. The main result of this article is a new upper bound for the expected distance, which is proved by combining a Fourier expansion with a truncation argument. Our bound matches the known rates for i.i.d. random variables up to logarithmic factors. In addition, we show how concentration inequalities around the mean can be obtained.
This work establishes fast rates of convergence for empirical barycenters over a large class of geodesic spaces with curvature bounds in the sense of Alexandrov. More specifically, we show that parametric rates of convergence are achievable under natural conditions that characterize the bi-extendibility of geodesics emanating from a barycenter. These results largely advance the state-of-the-art on the subject both in terms of rates of convergence and the variety of spaces covered. In particular, our results apply to infinite-dimensional spaces such as the 2-Wasserstein space, where bi-extendib…
We present a novel inference approach that we call Sample Out-of-Sample (or SOS) inference. The approach can be used widely, ranging from semi-supervised learning to stress testing, and it is fundamental in the application of data-driven Distributionally Robust Optimization (DRO). Our method enables measuring the impact of plausible out-of-sample scenarios in a given performance measure of interest, such as a financial loss. The methodology is inspired by Empirical Likelihood (EL), but we optimize the empirical Wasserstein distance (instead of the empirical likelihood) induced by observations.…
Consider the empirical measure, $\hat{\mathbb{P}}_N$, associated to $N$ i.i.d. samples of a given probability distribution $\mathbb{P}$ on the unit interval. For fixed $\mathbb{P}$ the Wasserstein distance between $\hat{\mathbb{P}}_N$ and $\mathbb{P}$ is a random variable on the sample space $[0,1]^N$. Our main result is that its normalised quantiles are asymptotically maximised when $\mathbb{P}$ is a convex combination between the uniform distribution supported on the two points $\{0,1\}$ and the uniform distribution on the unit interval $[0,1]$. This allows us to obtain explicit asymptotic c…
We study the problem of non-asymptotic deviations between a reference measure and its empirical version, in the 1-Wasserstein metric, under the standing assumption that the measure satisfies a transport-entropy inequality. We extend some results of F. Bolley, A. Guillin and C. Villani with simple proofs. Our methods are based on concentration inequalities and extend to the general setting of measures on a Polish space. Deviation bounds for the occupation measure of a Markov chain are also given, under the assumption that the chain is contractive on the space of Lipschitz functions. Throughout …
We show that several machine learning estimators, including square-root LASSO (Least Absolute Shrinkage and Selection) and regularized logistic regression can be represented as solutions to distributionally robust optimization (DRO) problems. The associated uncertainty regions are based on suitably defined Wasserstein distances. Hence, our representations allow us to view regularization as a result of introducing an artificial adversary that perturbs the empirical distribution to account for out-of-sample effects in loss estimation. In addition, we introduce RWPI (Robust Wasserstein Profile In…
Optimal transport (OT), and in particular the Wasserstein distance, has seen a surge of interest and applications in machine learning. However, empirical approximation under Wasserstein distances suffers from a severe curse of dimensionality, rendering them impractical in high dimensions. As a result, entropically regularized OT has become a popular workaround. However, while it enjoys fast algorithms and better statistical properties, it looses the metric structure that Wasserstein distances enjoy. This work proposes a novel Gaussian-smoothed OT (GOT) framework, that achieves the best of both…
In this paper, based on the Fr{é}chet mean, we define a notion of barycenter corresponding to a usual notion of statistical mean. We prove the existence of Wasserstein barycenters of random distributions defined on a geodesic space (E, d). We also prove the consistency of this barycenter in a general setting, that includes taking barycenters of empirical versions of the distributions or of a growing set of distributions.
Motivated by the statistical and computational challenges of computing Wasserstein distances in high-dimensional contexts, machine learning researchers have defined modified Wasserstein distances based on computing distances between one-dimensional projections of the measures. Different choices of how to aggregate these projected distances (averaging, random sampling, maximizing) give rise to different distances, requiring different statistical analyses. We define the \emph{Sliced Wasserstein Process}, a stochastic process defined by the empirical Wasserstein distance between projections of em…
Let $π\in Π(μ,ν)$ be a coupling between two probability measures $μ$ and $ν$ on a Polish space. In this article we propose and study a class of nonparametric measures of association between $μ$ and $ν$, which we call Wasserstein correlation coefficients. These coefficients are based on the Wasserstein distance between $ν$ and the disintegration $π_{x_1}$ of $π$ with respect to the first coordinate. We also establish basic statistical properties of this new class of measures: we develop a statistical theory for strongly consistent estimators and determine their convergence rate in the case of c…
Following the works of Berthet (1997), we first obtain exact clustering rates in the functional law of the iterated logarithm for the uniform empirical and quantile processes and for their increments. In a second time, we obtain functional Chung-type limit laws for the local empirical process for a class of target functions on the border of the Strassen set.
We propose of an improved version of the ubiquitous symmetrization inequality making use of the Wasserstein distance between a measure and its reflection in order to quantify the symmetry of the given measure. An empirical bound on this asymmetric correction term is derived through a bootstrap procedure and shown to give tighter results in practical settings than the original uncorrected inequality. Lastly, a wide range of applications are detailed including testing for data symmetry, constructing nonasymptotic high dimensional confidence sets, bounding the variance of an empirical process, an…
We estimate contrasts $\int_0 ^1 ρ(F^{-1}(u)-G^{-1}(u))du$ between two continuous distributions $F$ and $G$ on $\mathbb R$ such that the set $\{F=G\}$ is a finite union of intervals, possibly empty or $\mathbb{R}$. The non-negative convex cost function $ρ$ is not necessarily symmetric and the sample may come from any joint distribution $H$ on $\mathbb{R}^2$ with marginals $F$ and $G$ having light enough tails with respect to $ρ$. The rates of weak convergence and the limiting distributions are derived in a wide class of situations including the classical Wasserstein distances $W_1$ and $W_2$. …
Both Marcinkiewicz-Zygmund strong laws of large numbers (MZ-SLLNs) and ordinary strong laws of large numbers (SLLNs) for plug-in estimators of general statistical functionals are derived. It is used that if a statistical functional is "sufficiently regular", then a (MZ-) SLLN for the estimator of the unknown distribution function yields a (MZ-) SLLN for the corresponding plug-in estimator. It is in particular shown that many L-, V- and risk functionals are "sufficiently regular", and that known results on the strong convergence of the empirical process of α-mixing random variables can be impro…
Discrepancy measures between probability distributions, often termed statistical distances, are ubiquitous in probability theory, statistics and machine learning. To combat the curse of dimensionality when estimating these distances from data, recent work has proposed smoothing out local irregularities in the measured distributions via convolution with a Gaussian kernel. Motivated by the scalability of this framework to high dimensions, we investigate the structural and statistical behavior of the Gaussian-smoothed $p$-Wasserstein distance $\mathsf{W}_p^{(σ)}$, for arbitrary $p\geq 1$. After e…
In this paper, we aim to provide a statistical theory for object matching based on the Gromov-Wasserstein distance. To this end, we model general objects as metric measure spaces. Based on this, we propose a simple and efficiently computable asymptotic statistical test for pose invariant object discrimination. This is based on an empirical version of a $β$-trimmed lower bound of the Gromov-Wasserstein distance. We derive for $β\in[0,1/2)$ distributional limits of this test statistic. To this end, we introduce a novel $U$-type process indexed in $β$ and show its weak convergence. Finally, the t…
We develop a kernel projected Wasserstein distance for the two-sample test, an essential building block in statistics and machine learning: given two sets of samples, to determine whether they are from the same distribution. This method operates by finding the nonlinear mapping in the data space which maximizes the distance between projected distributions. In contrast to existing works about projected Wasserstein distance, the proposed method circumvents the curse of dimensionality more efficiently. We present practical algorithms for computing this distance function together with the non-asym…
Indian Institute of Technology Madras（印度理工学院马德拉斯分校）；Tata Consultancy Services Limited（塔塔咨询服务有限公司）
This paper presents a unified approach based on Wasserstein distance to derive concentration bounds for empirical estimates for two broad classes of risk measures defined in the paper. The classes of risk measures introduced include as special cases well known risk measures from the finance literature such as conditional value at risk (CVaR), optimized certainty equivalent risk, spectral risk measures, utility-based shortfall risk, cumulative prospect theory (CPT) value, rank dependent expected utility and distorted risk measures. Two estimation schemes are considered, one for each class of ri…
The smooth 1-Wasserstein distance (SWD) $W_1^σ$ was recently proposed as a means to mitigate the curse of dimensionality in empirical approximation while preserving the Wasserstein structure. Indeed, SWD exhibits parametric convergence rates and inherits the metric and topological structure of the classic Wasserstein distance. Motivated by the above, this work conducts a thorough statistical study of the SWD, including a high-dimensional limit distribution result for empirical $W_1^σ$, bootstrap consistency, concentration inequalities, and Berry-Esseen type bounds. The derived nondegenerate li…
