Let $f$ be a probability density and $C$ be an interval on which $f$ is bounded away from zero. By establishing the limiting distribution of the uniform error of the kernel estimates $f_n$ of $f$, Bickel and Rosenblatt (1973) provide confidence bands $B_n$ for $f$ on $C$ with asymptotic level $1-α\in]0,1[$. Each of the confidence intervals whose union gives $B_n$ has an asymptotic level equal to one; pointwise moderate deviations principles allow to prove that all these intervals share the same logarithmic asymptotic level. Now, as soon as both pointwise and uniform moderate deviations principles for $f_n$ exist, they share the same asymptotics. Taking this observation as a starting point, we present a new approach for the construction of confidence bands for $f$, based on the use of moderate deviations principles. The advantages of this approach are the following: (i) it enables to construct confidence bands, which have the same width (or even a smaller width) as the confidence bands provided by Bickel and Rosenblatt (1973), but which have a better aymptotic level; (ii) any confidence band constructed in that way shares the same logarithmic asymptotic level as all the confidence intervals, which make up this confidence band; (iii) it allows to deal with all the dimensions in the same way; (iv) it enables to sort out the problem of providing confidence bands for $f$ on compact sets on which $f$ vanishes (or on all $\bb R^d$), by introducing a truncating operation.
This paper develops bootstrap methods to construct uniform confidence bands for nonparametric spectral estimation of Lévy densities under high-frequency observations. We assume that we observe $n$ discrete observations at frequency $1/Δ> 0$, and work with the high-frequency setup where $Δ= Δ_{n} \to 0$ and $nΔ\to \infty$ as $n \to \infty$. We employ a spectral (or Fourier-based) estimator of the Lévy density, and develop novel implementations of Gaussian multiplier (or wild) and empirical (or Efron's) bootstraps to construct confidence bands for the spectral estimator on a compact set that doe…
The estimation of the Lévy density, the infinite-dimensional parameter controlling the jump dynamics of a Lévy process, is considered here under a discrete-sampling scheme. In this setting, the jumps are latent variables, the statistical properties of which can be assessed when the frequency and time horizon of observations increase to infinity at suitable rates. Nonparametric estimators for the Lévy density based on Grenander's method of sieves was proposed in Figueroa-López [IMS Lecture Notes 57 (2009) 117--146]. In this paper, central limit theorems for these sieve estimators, both pointwis…
Given a sample from some unknown continuous density $f:\mathbb{R}\to\mathbb{R}$, we construct adaptive confidence bands that are honest for all densities in a "generic" subset of the union of $t$-Hölder balls, $0<t\le r$, where $r$ is a fixed but arbitrary integer. The exceptional ("nongeneric") set of densities for which our results do not hold is shown to be nowhere dense in the relevant Hölder-norm topologies. In the course of the proofs we also obtain limit theorems for maxima of linear wavelet and kernel density estimators, which are of independent interest.
We study the problem of constructing honest and adaptive confidence intervals for the tail coefficient in the second order Pareto model, when the second order coefficient is unknown. This problem is translated into a testing problem on the second order parameter. By constructing an appropriate model and an associated test statistic, we provide a uniform and adaptive confidence interval for the first order parameter. We also provide an almost matching lower bound, which proves that the result is minimax optimal up to a logarithmic factor.
Data consisting of samples of probability density functions are increasingly prevalent, necessitating the development of methodologies for their analysis that respect the inherent nonlinearities associated with densities. In many applications, density curves appear as functional response objects in a regression model with vector predictors. For such models, inference is key to understand the importance of density-predictor relationships, and the uncertainty associated with the estimated conditional mean densities, defined as conditional Fréchet means under a suitable metric. Using the Wasserst…
We study nonparametric isotonic confidence intervals for monotone functions. In Banerjee and Wellner (2001) pointwise confidence intervals, based on likelihood ratio tests for the restricted and unrestricted MLE in the current status model, are introduced. We extend the method to the treatment of other models with monotone functions, and demonstrate our method by a new proof of the results in Banerjee and Wellner (2001) and also by constructing confidence intervals for monotone densities, for which still theory had to be developed. For the latter model we prove that the limit distribution of t…
We study the Nonparametric Maximum Likelihood Estimator (NPMLE) for estimating Gaussian location mixture densities in $d$-dimensions from independent observations. Unlike usual likelihood-based methods for fitting mixtures, NPMLEs are based on convex optimization. We prove finite sample results on the Hellinger accuracy of every NPMLE. Our results imply, in particular, that every NPMLE achieves near parametric risk (up to logarithmic multiplicative factors) when the true density is a discrete Gaussian mixture without any prior information on the number of mixture components. NPMLEs can natural…
We consider kernel estimation of marginal densities and regression functions of stationary processes. It is shown that for a wide class of time series, with proper centering and scaling, the maximum deviations of kernel density and regression estimates are asymptotically Gumbel. Our results substantially generalize earlier ones which were obtained under independence or beta mixing assumptions. The asymptotic results can be applied to assess patterns of marginal densities or regression functions via the construction of simultaneous confidence bands for which one can perform goodness-of-fit test…
This paper develops new methodology, together with related theories, for combining information from independent studies through confidence distributions. A formal definition of a confidence distribution and its asymptotic counterpart (i.e., asymptotic confidence distribution) are given and illustrated in the context of combining information. Two general combination methods are developed: the first along the lines of combining p-values, with some notable differences in regard to optimality of Bahadur type efficiency; the second by multiplying and normalizing confidence densities. The latter app…
Multi-armed bandit problems are considered as a paradigm of the trade-off between exploring the environment to find profitable actions and exploiting what is already known. In the stationary case, the distributions of the rewards do not change in time, Upper-Confidence Bound (UCB) policies have been shown to be rate optimal. A challenging variant of the MABP is the non-stationary bandit problem where the gambler must decide which arm to play while facing the possibility of a changing environment. In this paper, we consider the situation where the distributions of rewards remain constant over e…
In the density estimation model, we investigate the problem of constructing adaptive honest confidence sets with radius measured in Wasserstein distance $W_p$, $p\geq1$, and for densities with unknown regularity measured on a Besov scale. As sampling domains, we focus on the $d-$dimensional torus $\mathbb{T}^d$, in which case $1\leq p\leq 2$, and $\mathbb{R}^d$, for which $p=1$. We identify necessary and sufficient conditions for the existence of adaptive confidence sets with diameters of the order of the regularity-dependent $W_p$-minimax estimation rate. Interestingly, it appears that the po…
In this paper, we introduce a new estimator for the emission densities of a nonparametric hidden Markov model. It is adaptive and minimax with respect to each state's regularity--as opposed to globally minimax estimators, which adapt to the worst regularity among the emission densities. Our method is based on Goldenshluger and Lepski's methodology. It is computationally efficient and only requires a family of preliminary estimators, without any restriction on the type of estimators considered. We present two such estimators that allow to reach minimax rates up to a logarithmic term: a spectral…
We consider the problems of confidence estimation and hypothesis testing on a parameter of signal observed in Gaussian white noise. For these problems we point out lower bounds of asymptotic efficiency in the zone of moderate deviation probabilities. These lower bounds are versions of local asymptotic minimax Hajek-Le Cam lower bound in estimation and the lower bound for Pitman efficiency in hypothesis testing. The lower bounds were obtained for both logarithmic and sharp asymptotic of moderate deviation probabilities.
Amortized Bandwidth Learning for Kernel Density Estimation under Logarithmic Score
School of Mathematics, East China University of Science and Technology（华东理工大学数学学院）；Durham University（杜伦大学）；The London School of Economics and Political Science（伦敦政治经济学院）
The paper proposes chi-square and normal inference methodologies for the unknown coefficient matrix $B^*$ of size $p\times T$ in a Multi-Task (MT) linear model with $p$ covariates, $T$ tasks and $n$ observations under a row-sparse assumption on $B^*$. The row-sparsity $s$, dimension $p$ and number of tasks $T$ are allowed to grow with $n$. In the high-dimensional regime $p\ggg n$, in order to leverage row-sparsity, the MT Lasso is considered. We build upon the MT Lasso with a de-biasing scheme to correct for the bias induced by the penalty. This scheme requires the introduction of a new data-d…
Nonparametric statistics for distribution functions F or densities f=F' under qualitative shape constraints provides an interesting alternative to classical parametric or entirely nonparametric approaches. We contribute to this area by considering a new shape constraint: F is said to be bi-log-concave, if both log(F) and log(1 - F) are concave. Many commonly considered distributions are compatible with this constraint. For instance, any c.d.f. F with log-concave density f = F' is bi-log-concave. But in contrast to the latter constraint, bi-log-concavity allows for multimodal densities. We prov…
Modern construction of uniform confidence bands for nonparametric densities (and other functions) often relies on the classical Smirnov-Bickel-Rosenblatt (SBR) condition; see, for example, Giné and Nickl [Probab. Theory Related Fields 143 (2009) 569-596]. This condition requires the existence of a limit distribution of an extreme value type for the supremum of a studentized empirical process (equivalently, for the supremum of a Gaussian process with the same covariance function as that of the studentized empirical process). The principal contribution of this paper is to remove the need for thi…
We examine the integrated squared difference, also known as the L2 distance (L2D), between two probability densities. Such a distance metric allows for comparison of differences between pairs of distributions or changes in a distribution over time. We propose a targeted maximum likelihood estimator for this parameter based on samples of independent and identically distributed observations from both underlying distributions. We compare our method to kernel density estimation and demonstrate superior performance for our method with regards to confidence interval coverage rate and mean squared er…
We study an extension of the classic stochastic multi-armed bandit problem which involves multiple plays and Markovian rewards in the rested bandits setting. In order to tackle this problem we consider an adaptive allocation rule which at each stage combines the information from the sample means of all the arms, with the Kullback-Leibler upper confidence bound of a single arm which is selected in round-robin way. For rewards generated from a one-parameter exponential family of Markov chains, we provide a finite-time upper bound for the regret incurred from this adaptive allocation rule, which …
We develop honest and locally adaptive confidence bands for probability densities. They provide substantially improved confidence statements in case of inhomogeneous smoothness, and are easily implemented and visualized. The article contributes conceptual work on locally adaptive inference as a straightforward modification of the global setting imposes severe obstacles for statistical purposes. Among others, we introduce a statistical notion of local Hölder regularity and prove a correspondingly strong version of local adaptivity. We substantially relax the straightforward localization of the …
We consider the estimation of two-sample integral functionals, of the type that occur naturally, for example, when the object of interest is a divergence between unknown probability densities. Our first main result is that, in wide generality, a weighted nearest neighbour estimator is efficient, in the sense of achieving the local asymptotic minimax lower bound. Moreover, we also prove a corresponding central limit theorem, which facilitates the construction of asymptotically valid confidence intervals for the functional, having asymptotically minimal width. One interesting consequence of our …
Finding anonymization mechanisms to protect personal data is at the heart of recent machine learning research. Here, we consider the consequences of local differential privacy constraints on goodness-of-fit testing, i.e. the statistical problem assessing whether sample points are generated from a fixed density $f_0$, or not. The observations are kept hidden and replaced by a stochastic transformation satisfying the local differential privacy constraint. In this setting, we propose a testing procedure which is based on an estimation of the quadratic distance between the density $f$ of the unobs…
We present a unified technique for sequential estimation of convex divergences between distributions, including integral probability metrics like the kernel maximum mean discrepancy, $φ$-divergences like the Kullback-Leibler divergence, and optimal transport costs, such as powers of Wasserstein distances. This is achieved by observing that empirical convex divergences are (partially ordered) reverse submartingales with respect to the exchangeable filtration, coupled with maximal inequalities for such processes. These techniques appear to be complementary and powerful additions to the existing …
Regret Lower Bound for Online Learning of Quantum States with Logarithmic Loss, with Applications to Quantum State Tomography and Confidence Sequences for Matrix Means
National Taiwan University（台湾大学）；Center for Quantum Science and Engineering, National Taiwan University（台湾大学量子科学与工程中心）
This paper develops a method to construct uniform confidence bands in deconvolution when the error distribution is unknown. We mainly focus on the baseline setting where an auxiliary sample from the error distribution is available and the error density is ordinary smooth. The auxiliary sample may directly come from validation data, or can be constructed from panel data with a symmetric error distribution. We also present extensions of the results on confidence bands to the case of super-smooth error densities. Simulation studies demonstrate the performance of the multiplier bootstrap confidenc…
