A weighted Shiryaev-Roberts change detection procedure is shown to approximately minimize the expected delay to detection as well as higher moments of the detection delay among all change-point detection procedures with the given low maximal local probability of a false alarm within a window of a fixed length in pointwise and minimax settings for general non-i.i.d. data models and for the composite post-change hypothesis when the post-change parameter is unknown. We establish very general conditions for the models under which the weighted Shiryaev-Roberts procedure is asymptotically optimal. These conditions are formulated in terms of the rate of convergence in the strong law of large numbers for the log-likelihood ratios between the "change" and "no-change" hypotheses, and we also provide sufficient conditions for a large class of ergodic Markov processes. Examples, where these conditions hold, are given.
We consider the quickest change-point detection problem in pointwise and minimax settings for general dependent data models. Two new classes of sequential detection procedures associated with the maximal "local" probability of a false alarm within a period of some fixed length are introduced. For these classes of detection procedures, we consider two popular risks: the expected positive part of the delay to detection and the conditional delay to detection. Under very general conditions for the observations, we show that the popular Shiryaev--Roberts procedure is asymptotically optimal, as the …
The nonparametric regression with a random design model is considered. We want to recover the regression function at a point x where the design density is vanishing or exploding. Depending on assumptions on the regression function local regularity and on the design local behaviour, we find several minimax rates. These rates lie in a wide range, from slow l(n) rates where l(.) is slowly varying (for instance (log n)^(-1)) to fast n^(-1/2) * l(n) rates. If the continuity modulus of the regression function at x can be bounded from above by a s-regularly varying function, and if the design density…
We consider the estimation of a density at a fixed point under a local differential privacy constraint, where the observations are anonymised before being available for statistical inference. We propose both a privatised version of a projection density estimator as well as a kernel density estimator and derive their minimax rates under a privacy constraint. There is a twofold deterioration of the minimax rates due to the anonymisation, which we show to be unavoidable by providing lower bounds. In both estimation procedures a tuning parameter has to be chosen. We suggest a variant of the classi…
We study a class of statistical inverse problems with non-linear pointwise operators motivated by concrete statistical applications. A two-step procedure is proposed, where the first step smoothes the data and inverts the non-linearity. This reduces the initial non-linear problem to a linear inverse problem with deterministic noise, which is then solved in a second step. The noise reduction step is based on wavelet thresholding and is shown to be minimax optimal (up to logarithmic factors) in a pointwise function-dependent sense. Our analysis is based on a modified notion of Hölder smoothness …
In this paper, we study the problem of pointwise estimation of a multivariate density. We provide a data-driven selection rule from the family of kernel estimators and derive for it a pointwise oracle inequality. Using the latter bound, we show that the proposed estimator is minimax and minimax adaptive over the scale of anisotropic Nikolskii classes. It is important to emphasize that our estimation method adjusts automatically to eventual independence structure of the underlying density. This, in its turn, allows to reduce significantly the influence of the dimension on the accuracy of estima…
We consider the problem of testing a particular type of composite null hypothesis under a nonparametric multivariate regression model. For a given quadratic functional $Q$, the null hypothesis states that the regression function $f$ satisfies the constraint $Q[f]=0$, while the alternative corresponds to the functions for which $Q[f]$ is bounded away from zero. On the one hand, we provide minimax rates of testing and the exact separation constants, along with a sharp-optimal testing procedure, for diagonal and nonnegative quadratic functionals. We consider smoothness classes of ellipsoidal form…
We consider the nonparametric regression with a random design model, and we are interested in the adaptive estimation of the regression at a point $x\_0$ where the design is degenerate. When the design density is $β$-regularly varying at $x\_0$ and $f$ has a smoothness $s$ in the Hölder sense, we know from Gaïffas (2004)\nocite{gaiffas04a} that the minimax rate is equal to $n^{-s/(1+2s+β)} \ell(1/n)$ where $\ell$ is slowly varying. In this paper we provide an estimator which is adaptive both on the design and the regression function smoothness and we show that it converges with the rate $(\log…
This paper is concerned with adaptive kernel estimation of the Lévy density N(x) for bounded-variation pure-jump Lévy processes. The sample path is observed at n discrete instants in the "high frequency" context (Δ= Δ(n) tends to zero while nΔtends to infinity). We construct a collection of kernel estimators of the function g(x)=xN(x) and propose a method of local adaptive selection of the bandwidth. We provide an oracle inequality and a rate of convergence for the quadratic pointwise risk. This rate is proved to be the optimal minimax rate. We give examples and simulation results for processe…
We estimate the quantum state of a light beam from results of quantum homodyne measurements performed on identically prepared quantum systems. The state is represented through the Wigner function, a generalized probability density on $\mathbb{R}^2$ which may take negative values and must respect intrinsic positivity constraints imposed by quantum physics. The effect of the losses due to detection inefficiencies, which are always present in a real experiment, is the addition to the tomographic data of independent Gaussian noise. We construct a kernel estimator for the Wigner function, prove tha…
This paper deals with the nonparametric estimation in heteroscedastic regression $ Y_i=f(X_i)+ξ_i, \: i=1,...,n $, with incomplete information, i.e. each real random variable $ ξ_i $ has a density $ g_{i} $ which is unknown to the statistician. The aim is to estimate the regression function $ f $ at a given point. Using a local polynomial fitting from M-estimator denoted $ \hat f^h $ and applying Lepski's procedure for the bandwidth selection, we construct an estimator $ \hat f^{\hat h} $ which is adaptive over the collection of isotropic Hölder classes. In particular, we establish new exponen…
We revisit the classical problem of nonparametric density estimation but impose local differential privacy constraints. Under such constraints, the original multivariate data $X_1,\ldots,X_n \in \mathbb{R}^d$ cannot be directly observed, and all estimators are functions of the randomised output of a suitable privacy mechanism. The statistician is free to choose the form of the privacy mechanism, and in this work we propose to add Laplace distributed noise to a discretisation of the location of a vector $X_i$. Based on these randomised data, we design a novel estimator of the density function, …
The classical binary hypothesis testing problem is revisited. We notice that when one of the hypotheses is composite, there is an inherent difficulty in defining an optimality criterion that is both informative and well-justified. For testing in the simple normal location problem (that is, testing for the mean of multivariate Gaussians), we overcome the difficulty as follows. In this problem there exists a natural hardness order between parameters as for different parameters the error-probailities curves (when the parameter is known) are either identical, or one dominates the other. We can thu…
This paper considers the problem of joint change detection and identification assuming multiple composite postchange hypotheses. We propose a multihypothesis changepoint detection-identification procedure that controls the probabilities of false alarm and wrong identification. We show that the proposed procedure is asymptotically minimax and pointwise optimal, minimizing moments of the detection delay as probabilities of false alarm and wrong identification approach zero. The asymptotic optimality properties hold for general stochastic models with dependent observations. We illustrate general …
Since Stein's 1956 seminal paper, shrinkage has played a fundamental role in both parametric and nonparametric inference. This article discusses minimaxity and adaptive minimaxity in nonparametric function estimation. Three interrelated problems, function estimation under global integrated squared error, estimation under pointwise squared error, and nonparametric confidence intervals, are considered. Shrinkage is pivotal in the development of both the minimax theory and the adaptation theory. While the three problems are closely connected and the minimax theories bear some similarities, the ad…
We consider the problem of testing hypotheses on the copula density from $n$ bi-dimensional observations. We wish to test the null hypothesis characterized by a parametric class against a composite nonparametric alternative. Each density under the alternative is separated in the $L_2$-norm from any density lying in the null hypothesis. The copula densities under consideration are supposed to belong to a range of Besov balls. According to the minimax approach, the testing problem is solved in an adaptive framework: it leads to a $\log\log$ term loss in the minimax rate of testing in comparison …
The paper deals with minimax optimal statistical tests for two composite hypotheses, where each hypothesis is defined by a non-parametric uncertainty set of feasible distributions. It is shown that for every pair of uncertainty sets of the f-divergence ball type, a pair of uncertainty sets of the density band type can be constructed, which is equivalent in the sense that it admits the same pair of least favorable distributions. This result implies that robust tests under $f$-divergence ball uncertainty, which are typically only minimax optimal for the single sample case, are also fixed sample …
We consider the problem of sparsity testing in the high-dimensional linear regression model. The problem is to test whether the number of non-zero components (aka the sparsity) of the regression parameter $θ^*$ is less than or equal to $k_0$. We pinpoint the minimax separation distances for this problem, which amounts to quantifying how far a $k_1$-sparse vector $θ^*$ has to be from the set of $k_0$-sparse vectors so that a test is able to reject the null hypothesis with high probability. Two scenarios are considered. In the independent scenario, the covariates are i.i.d. normally distributed …
This paper focuses on recursive estimation of time varying autoregressive processes in a nonparametric setting. The stability of the model is revisited and uniform results are provided when the time-varying autoregressive parameters belong to appropriate smoothness classes. An adequate normalization for the correction term used in the recursive estimation procedure allows for very mild assumptions on the innovations distributions. The rate of convergence of the pointwise estimates is shown to be minimax in $β$-Lipschitz classes for $0<β\leq1$. For $1<β\leq 2$, this property no longer holds. Th…
In this paper, we study the problem of pointwise estimation of a multivariate function. We develop a general pointwise estimation procedure that is based on selection of estimators from a large parameterized collection. An upper bound on the pointwise risk is established and it is shown that the proposed selection procedure specialized for different collections of estimators leads to minimax and adaptive minimax estimators in various settings.
We consider the goodness-of fit testing problem for Hölder smooth densities over $\mathbb{R}^d$: given $n$ iid observations with unknown density $p$ and given a known density $p_0$, we investigate how large $ρ$ should be to distinguish, with high probability, the case $p=p_0$ from the composite alternative of all Hölder-smooth densities $p$ such that $\|p-p_0\|_t \geq ρ$ where $t \in [1,2]$. The densities are assumed to be defined over $\mathbb{R}^d$ and to have Hölder smoothness parameter $α>0$. In the present work, we solve the case $α\leq 1$ and handle the case $α>1$ using an additional tec…
This paper studies density estimation under pointwise loss in the setting of contamination model. The goal is to estimate $f(x_0)$ at some $x_0\in\mathbb{R}$ with i.i.d. observations, $$ X_1,\dots,X_n\sim (1-ε)f+εg, $$ where $g$ stands for a contamination distribution. In the context of multiple testing, this can be interpreted as estimating the null density at a point. We carefully study the effect of contamination on estimation through the following model indices: contamination proportion $ε$, smoothness of target density $β_0$, smoothness of contamination density $β_1$, and level of contami…
We consider the goodness-of-fit testing problem of distinguishing whether the data are drawn from a specified distribution, versus a composite alternative separated from the null in the total variation metric. In the discrete case, we consider goodness-of-fit testing when the null distribution has a possibly growing or unbounded number of categories. In the continuous case, we consider testing a Lipschitz density, with possibly unbounded support, in the low-smoothness regime where the Lipschitz parameter is not assumed to be constant. In contrast to existing results, we show that the minimax r…
In this paper we study the problem of pointwise density estimation from observations with multiplicative measurement errors. We elucidate the main feature of this problem: the influence of the estimation point on the estimation accuracy. In particular, we show that, depending on whether this point is separated away from zero or not, there are two different regimes in terms of the rates of convergence of the minimax risk. In both regimes we develop kernel--type density estimators and prove upper bounds on their maximal risk over suitable nonparametric classes of densities. We show that the prop…
We consider non-parametric density estimation in the framework of local approximate differential privacy. In contrast to centralized privacy scenarios with a trusted curator, in the local setup anonymization must be guaranteed already on the individual data owners' side and therefore must precede any data mining tasks. Thus, the published anonymized data should be compatible with as many statistical procedures as possible. We suggest adding Laplace noise and Gaussian processes (both appropriately scaled) to kernel density estimators to obtain approximate differential private versions of the la…
The estimation of the Lévy density, the infinite-dimensional parameter controlling the jump dynamics of a Lévy process, is considered here under a discrete-sampling scheme. In this setting, the jumps are latent variables, the statistical properties of which can be assessed when the frequency and time horizon of observations increase to infinity at suitable rates. Nonparametric estimators for the Lévy density based on Grenander's method of sieves was proposed in Figueroa-López [IMS Lecture Notes 57 (2009) 117--146]. In this paper, central limit theorems for these sieve estimators, both pointwis…
In this article, the problem of semi-parametric inference on the parameters of a multidimensional Lévy process $L_t$ with independent components based on the low-frequency observations of the corresponding time-changed Lévy process $L_{\mathcal{T}(t)}$, where $\mathcal{T}$ is a nonnegative, nondecreasing real-valued process independent of $L_t$, is studied. We show that this problem is closely related to the problem of composite function estimation that has recently gotten much attention in statistical literature. Under suitable identifiability conditions, we propose a consistent estimate for …
We consider two nonparametric procedures for estimating a concave distribution function based on data corrupted with additive noise generated by a bounded decreasing density on $(0,\infty)$. For the maximum likelihood (ML) estimator and least squares (LS) estimator, we state qualitative properties, prove consistency and propose a computational algorithm. For the LS estimator and its derivative, we also derive the pointwise asymptotic distribution. Moreover, the rate $n^{-2/5}$ achieved by the LS estimator is shown to be minimax for estimating the distribution function at a fixed point.
We study the problem of nonparametric estimation of a multivariate function $g:\mathbb {R}^d\to\mathbb{R}$ that can be represented as a composition of two unknown smooth functions $f:\mathbb{R}\to\mathbb{R}$ and $G:\mathbb{R}^d\to \mathbb{R}$. We suppose that $f$ and $G$ belong to known smoothness classes of functions, with smoothness $γ$ and $β$, respectively. We obtain the full description of minimax rates of estimation of $g$ in terms of $γ$ and $β$, and propose rate-optimal estimators for the sup-norm loss. For the construction of such estimators, we first prove an approximation result for…
In this article we propose a locally adaptive strategy for estimating a function from its Exponential Radon Transform (ERT) data, without prior knowledge of the smoothness of functions that are to be estimated. We build a non-parametric kernel type estimator and show that for a class of functions comprising a wide Sobolev regularity scale, our proposed strategy follows the minimax optimal rate up to a $\log{n}$ factor. We also show that there does not exist an optimal adaptive estimator on the Sobolev scale when the pointwise risk is used and in fact the rate achieved by the proposed estimator…
The problem of quickest change detection is studied, where there is an additional constraint on the cost of observations used before the change point and where the post-change distribution is composite. Minimax formulations are proposed for this problem. It is assumed that the post-change family of distributions has a member which is least favorable in some sense. An algorithm is proposed in which on-off observation control is employed using the least favorable distribution, and a generalized likelihood ratio based approach is used for change detection. Under the additional condition that eith…
We study the rates of estimation of finite mixing distributions, that is, the parameters of the mixture. We prove that under some regularity and strong identifiability conditions, around a given mixing distribution with $m_0$ components, the optimal local minimax rate of estimation of a mixing distribution with $m$ components is $n^{-1/(4(m-m_0) + 2)}$. This corrects a previous paper by Chen (1995) in The Annals of Statistics. By contrast, it turns out that there are estimators with a (non-uniform) pointwise rate of estimation of $n^{-1/2}$ for all mixing distributions with a finite number of …
Based on two independent samples X_1,...,X_m and X_{m+1},...,X_n drawn from multivariate distributions with unknown Lebesgue densities p and q respectively, we propose an exact multiple test in order to identify simultaneously regions of significant deviations between p and q. The construction is built from randomized nearest-neighbor statistics. It does not require any preliminary information about the multivariate densities such as compact support, strict positivity or smoothness and shape properties. The properly adjusted multiple testing procedure is shown to be sharp-optimal for typical a…
A robust minimax test for two composite hypotheses, which are determined by the neighborhoods of two nominal distributions with respect to a set of distances - called $α-$divergence distances, is proposed. Sion's minimax theorem is adopted to characterize the saddle value condition. Least favorable distributions, the robust decision rule and the robust likelihood ratio test are derived. If the nominal probability distributions satisfy a symmetry condition, the design procedure is shown to be simplified considerably. The parameters controlling the degree of robustness are bounded from above and…
We consider the problem of estimating the value l(ϕ) of a linear functional, where the structural function ϕ models a nonparametric relationship in presence of instrumental variables. We propose a plug-in estimator which is based on a dimension reduction technique and additional thresholding. It is shown that this estimator is consistent and can attain the minimax optimal rate of convergence under additional regularity conditions. This, however, requires an optimal choice of the dimension parameter m depending on certain characteristics of the structural function ϕ and the joint distribution o…
We consider composite-composite testing problems for the expectation in the Gaussian sequence model where the null hypothesis corresponds to a convex subset $\mathcal{C}$ of $\mathbb{R}^d$. We adopt a minimax point of view and our primary objective is to describe the smallest Euclidean distance between the null and alternative hypotheses such that there is a test with small total error probability. In particular, we focus on the dependence of this distance on the dimension $d$ and the sample size/variance parameter $n$ giving rise to the minimax separation rate. In this paper we discuss lower …
University of California Santa Barbara（加州大学圣巴巴拉分校）
Machine learning models are vulnerable to adversarial attacks that can often cause misclassification by introducing small but well designed perturbations. In this paper, we explore, in the setting of classical composite hypothesis testing, a defense strategy based on the generalized likelihood ratio test (GLRT), which jointly estimates the class of interest and the adversarial perturbation. We evaluate the GLRT approach for the special case of binary hypothesis testing in white Gaussian noise under $\ell_{\infty}$ norm-bounded adversarial perturbations, a setting for which a minimax strategy o…
We propose an estimation procedure for linear functionals based on Gaussian model selection techniques. We show that the procedure is adaptive, and we give a non asymptotic oracle inequality for the risk of the selected estimator with respect to the $\mathbb{L}_p$ loss. An application to the problem of estimating a signal or its $r^{th}$ derivative at a given point is developed and minimax rates are proved to hold uniformly over Besov balls. We also apply our non asymptotic oracle inequality to the estimation of the mean of the signal on an interval with length depending on the noise level. Si…
A truncated sequential procedure is constructed for estimating the drift coefficient at a given state point based on discrete data of ergodic diffusion process. A nonasymptotic upper bound is obtained for a pointwise absolute error risk. The optimal convergence rate and a sharp constant in the bounds are found for the asymptotic pointwise minimax risk. As a consequence, the efficiency is obtained of the proposed sequential procedure.
In this paper, we consider the problem of estimating a conditional density in moderately large dimensions. Much more informative than regression functions, conditional densities are of main interest in recent methods, particularly in the Bayesian framework (studying the posterior distribution, finding its modes...). Considering a recently studied family of kernel estimators, we select a pointwise multivariate bandwidth by revisiting the greedy algorithm Rodeo (Regularisation Of Derivative Expectation Operator). The method addresses several issues: being greedy and computationally efficient by …
A general lower bound is developed for the minimax risk when estimating an arbitrary functional. The bound is based on testing two composite hypotheses and is shown to be effective in estimating the nonsmooth functional ${\frac{1}{n}}\sum|θ_i|$ from an observation $Y\sim N(θ,I_n)$. This problem exhibits some features that are significantly different from those that occur in estimating conventional smooth functionals. This is a setting where standard techniques fail to yield sharp results. A sharp minimax lower bound is established by applying the general lower bound technique based on testing …
This paper is devoted to the estimation of the common marginal density function of weakly dependent processes. The accuracy of estimation is measured using pointwise risks. We propose a datadriven procedure using kernel rules. The bandwidth is selected using the approach of Goldenshluger and Lepski and we prove that the resulting estimator satisfies an oracle type inequality. The procedure is also proved to be adaptive (in a minimax framework) over a scale of Hölder balls for several types of dependence: stong mixing processes, $λ$-dependent processes or i.i.d. sequences can be considered usin…
University of California Santa Barbara（加州大学圣巴巴拉分校）
Machine learning models are known to be susceptible to adversarial attacks which can cause misclassification by introducing small but well designed perturbations. In this paper, we consider a classical hypothesis testing problem in order to develop fundamental insight into defending against such adversarial perturbations. We interpret an adversarial perturbation as a nuisance parameter, and propose a defense based on applying the generalized likelihood ratio test (GLRT) to the resulting composite hypothesis testing problem, jointly estimating the class of interest and the adversarial perturbat…
