We consider the problem of estimating the density $g$ of identically distributed variables $X\_i$, from a sample $Z\_1, ..., Z\_n$ where $Z\_i=X\_i+σε\_i$, $i=1, ..., n$ and $σε\_i$ is a noise independent of $X\_i$ with known density $ σ^{-1}f\_ε(./σ)$. We generalize adaptive estimators, constructed by a model selection procedure, described in Comte et al. (2005). We study numerically their properties in various contexts and we test their robustness. Comparisons are made with respect to deconvolution kernel estimators, misspecification of errors, dependency,... It appears that our estimation algorithm, based on a fast procedure, performs very well in all contexts.
The authors consider the problem of estimating the density $g$ of independent and identically distributed variables $X\_i$, from a sample $Z\_1, ..., Z\_n$ where $Z\_i=X\_i+σε\_i$, $i=1, ..., n$, $ε$ is a noise independent of $X$, with $σε$ having known distribution. They present a model selection procedure allowing to construct an adaptive estimator of $g$ and to find non-asymptotic bounds for its $\mathbb{L}\_2(\mathbb{R})$-risk. The estimator achieves the minimax rate of convergence, in most cases where lowers bounds are available. A simulation study gives an illustration of the good practi…
Quantile estimation in deconvolution problems is studied comprehensively. In particular, the more realistic setup of unknown error distributions is covered. Our plug-in method is based on a deconvolution density estimator and is minimax optimal under minimal and natural conditions. This closes an important gap in the literature. Optimal adaptive estimation is obtained by a data-driven bandwidth choice. As a side result, we obtain optimal rates for the plug-in estimation of distribution functions with unknown error distributions. The method is applied to a real data example.
In this paper we consider a statistical estimation problem known as atomic deconvolution. Introduced in reliability, this model has a direct application when considering biological data produced by flow cytometers. In these experiments, biologists measure the fluorescence emission of treated cells and compare them with their natural emission to study the presence of specific molecules on the cells' surface. They observe a signal which is composed of a noise (the natural fluorescence) plus some additional signal related to the quantity of molecule present on the surface if any. From a statistic…
In this paper, we address the problem of estimating a multidimensional density $f$ by using indirect observations from the statistical model $Y=X+\varepsilon$. Here, $\varepsilon$ is a measurement error independent of the random vector $X$ of interest, and having a known density with respect to the Lebesgue measure. Our aim is to obtain optimal accuracy of estimation under $L_p$-losses when the error $\varepsilon$ has a characteristic function with a polynomial decay. To achieve this goal, we first construct a kernel estimator of $f$ which is fully data driven. Then, we derive for it an oracle…
In the present paper we consider Laplace deconvolution for discrete noisy data observed on the interval whose length may increase with a sample size. Although this problem arises in a variety of applications, to the best of our knowledge, it has been given very little attention by the statistical community. Our objective is to fill this gap and provide statistical treatment of Laplace deconvolution problem with noisy discrete data. The main contribution of the paper is explicit construction of an asymptotically rate-optimal (in the minimax sense) Laplace deconvolution estimator which is adapti…
We consider a circular deconvolution problem, in which the density $f$ of a circular random variable $X$ must be estimated nonparametrically based on an i.i.d. sample from a noisy observation $Y$ of $X$. The additive measurement error is supposed to be independent of $X$. The objective of this work was to construct a fully data-driven estimation procedure when the error density $φ$ is unknown. We assume that in addition to the i.i.d. sample from $Y$, we have at our disposal an additional i.i.d. sample drawn independently from the error distribution. We first develop a minimax theory in terms o…
We consider the problem of denoising a function observed after a convolution with a random filter independent of the noise and satisfying some mean smoothness condition depending on an ill posedness coefficient. We establish the minimax rates for the Lp risk over balls of periodic Besov spaces with respect to the level of noise, and we provide an adaptive estimator achieving these rates up to log factors. Simulations were performed to highlight the effects of the ill posedness and of the distribution of the filter on the efficiency of the estimator.
We consider quantile regression processes from censored data under dependent data structures and derive a uniform Bahadur representation for those processes. We also consider cases where the dimension of the parameter in the quantile regression model is large. It is demonstrated that traditional penalized estimators such as the adaptive lasso yield sub-optimal rates if the coefficients of the quantile regression cross zero. New penalization techniques are introduced which are able to deal with specific problems of censored data and yield estimates with an optimal rate. In contrast to most of t…
This paper considers the deconvolution problem in the case where the target signal is multidimensional and no information is known about the noise distribution. More precisely, no assumption is made on the noise distribution and no samples are available to estimate it: the deconvolution problem is solved based only on the corrupted signal observations. We establish the identifiability of the model up to translation when the signal has a Laplace transform with an exponential growth smaller than $2$ and when it can be decomposed into two dependent components. Then, we propose an estimator of the…
The subject of this paper is the problem of nonparametric estimation of a continuous distribution function from observations with measurement errors. We study minimax complexity of this problem when unknown distribution has a density belonging to the Sobolev class, and the error density is ordinary smooth. We develop rate optimal estimators based on direct inversion of empirical characteristic function. We also derive minimax affine estimators of the distribution function which are given by an explicit convex optimization problem. Adaptive versions of these estimators are proposed, and some nu…
This paper considers the problem of adaptive estimation of a mean pattern in a randomly shifted curve model. We show that this problem can be transformed into a linear inverse problem, where the density of the random shifts plays the role of a convolution operator. An adaptive estimator of the mean pattern, based on wavelet thresholding is proposed. We study its consistency for the quadratic risk as the number of observed curves tends to infinity, and this estimator is shown to achieve a near-minimax rate of convergence over a large class of Besov balls. This rate depends both on the smoothnes…
This article deals with adaptive nonparametric estimation for Lévy processes observed at low frequency. For general linear functionals of the Lévy measure, we construct kernel estimators, provide upper risk bounds and derive rates of convergence under regularity assumptions. Our focus lies on the adaptive choice of the bandwidth, using model selection techniques. We face here a non-standard problem of model selection with unknown variance. A new approach towards this problem is proposed, which also allows a straightforward generalization to a classical density deconvolution framework.
This paper considers the problem of adaptive estimation of a non-homogeneous intensity function from the observation of n independent Poisson processes having a common intensity that is randomly shifted for each observed trajectory. We show that estimating this intensity is a deconvolution problem for which the density of the random shifts plays the role of the convolution operator. In an asymptotic setting where the number n of observed trajectories tends to infinity, we derive upper and lower bounds for the minimax quadratic risk over Besov balls. Non-linear thresholding in a Meyer wavelet b…
This paper is concerned with nonparametric estimation of the Lévy density of a pure jump Lévy process. The sample path is observed at $n$ discrete instants with fixed sampling interval. We construct a collection of estimators obtained by deconvolution methods and deduced from appropriate estimators of the characteristic function and its first derivative. We obtain a bound for the ${\mathbb L}^2$-risk, under general assumptions on the model. Then we propose a penalty function that allows to build an adaptive estimator. The risk bound for the adaptive estimator is obtained under additional assum…
We construct an adaptive wavelet estimator that attains minimax near-optimal rates in a wide range of Besov balls. The convergence rates are affected only by the weakest dependence amongst the channels, and take into account both noise sources.
A novel approach to quantile estimation in multivariate linear regression models with change-points is proposed: the change-point detection and the model estimation are both performed automatically, by adopting either the quantile fused penalty or the adaptive version of the quantile fused penalty. These two methods combine the idea of the check function used for the quantile estimation and the $L_1$ penalization principle known from the signal processing and, unlike some standard approaches, the presented methods go beyond typical assumptions usually required for the model errors, such as sub…
Bayesian density deconvolution using nonparametric prior distributions is a useful alternative to the frequentist kernel based deconvolution estimators due to its potentially wide range of applicability, straightforward uncertainty quantification and generalizability to more sophisticated models. This article is the first substantive effort to theoretically quantify the behavior of the posterior in this recent line of research. In particular, assuming a known supersmooth error density, a Dirichlet process mixture of Normals on the true density leads to a posterior convergence rate same as the …
We introduce a new procedure to select the optimal cutoff parameter for Fourier density estimators that leads to adaptive rate optimal estimators, up to a logarithmic factor. This adaptive procedure applies for different inverse problems. We illustrate it on two classical examples: deconvolution and decompounding, i.e. non-parametric estimation of the jump density of a compound Poisson process from the observation of n increments of length $Δ$ > 0. For this latter example, we first build an estimator for which we provide an upper bound for its L 2-risk that is valid simultaneously for sampling…
It is common, in deconvolution problems, to assume that the measurement errors are identically distributed. In many real-life applications, however, this condition is not satisfied and the deconvolution estimators developed for homoscedastic errors become inconsistent. In this paper, we introduce a kernel estimator of a density in the case of heteroscedastic contamination. We establish consistency of the estimator and show that it achieves optimal rates of convergence under quite general conditions. We study the limits of application of the procedure in some extreme situations, where we show t…
In the convolution model $Z\_i=X\_i+ ε\_i$, we give a model selection procedure to estimate the density of the unobserved variables $(X\_i)\_{1 \leq i \leq n}$, when the sequence $(X\_i)\_{i \geq 1}$ is strictly stationary but not necessarily independent. This procedure depends on wether the density of $ε\_i$ is super smooth or ordinary smooth. The rates of convergence of the penalized contrast estimators are the same as in the independent framework, and are minimax over most classes of regularity on ${\mathbb R}$. Our results apply to mixing sequences, but also to many other dependent sequenc…
Asgl: A Python Package for Penalized Linear and Quantile Regression
University of Columbia（哥伦比亚大学）；uc3m-Santander Big Data Institute（uc3m-桑坦德大数据研究所）；Universitat Politècnica de València（瓦伦西亚理工大学）；Universidad Carlos III de Madrid（马德里卡洛斯三世大学）
The problem of estimating the shift (or, equivalently, the center of symmetry) of an unknown symmetric and periodic function $f$ observed in Gaussian white noise is considered. Using the blockwise Stein method, a penalized profile likelihood with a data-driven penalization is introduced so that the estimator of the center of symmetry is defined as the maximizer of the penalized profile likelihood. This estimator has the advantage of being independent of the functional class to which the signal $f$ si assumed to belong and, furthermore, is shown to be semiparametrically adaptive and efficient. …
We propose a ridge-penalized adaptive Mantel test (AdaMant) for evaluating the association of two high-dimensional sets of features. By introducing a ridge penalty, AdaMant tests the association across many metrics simultaneously. We demonstrate how ridge penalization bridges Euclidean and Mahalanobis distances and their corresponding linear models from the perspective of association measurement and testing. This result is not only theoretically interesting but also has important implications in penalized hypothesis testing, especially in high dimensional settings such as imaging genetics. App…
In the present paper, we consider the estimation of a periodic two-dimensional function $f(\cdot,\cdot)$ based on observations from its noisy convolution, and convolution kernel $g(\cdot,\cdot)$ unknown. We derive the minimax lower bounds for the mean squared error assuming that $f$ belongs to certain Besov space and the kernel function $g$ satisfies some smoothness properties. We construct an adaptive hard-thresholding wavelet estimator that is asymptotically near-optimal within a logarithmic factor in a wide range of Besov balls. The proposed estimation algorithm implements a truncation to e…
We investigate the problem of estimating a function $f$ based on observations from its noisy convolution when the noise exhibits long-range dependence. We construct an adaptive estimator based on the kernel method, derive minimax lower bound for the $L^2$-risk when $f$ belongs to Sobolev space and show that such estimator attains optimal rates that deteriorate as the LRD worsens.
It is a typical standard assumption in the density deconvolution problem that the characteristic function of the measurement error distribution is non-zero on the real line. While this condition is assumed in the majority of existing works on the topic, there are many problem instances of interest where it is violated. In this paper we focus on non--standard settings where the characteristic function of the measurement errors has zeros, and study how zeros multiplicity affects the estimation accuracy. For a prototypical problem of this type we demonstrate that the best achievable estimation ac…
We investigate minimax results for the anisotropic functional deconvolution model when observations are affected by the presence of long-memory. Under specific conditions about the covariance matrices of the errors, we follow a standard procedure to construct an adaptive wavelet-based estimator that attains asymptotically near-optimal convergence rates. These rates depend on the parameter associated with the weakest long-range dependence, and deteriorate as the intensity of long-memory increases. This behavior suggests that the estimator adjusts to the best case scenario and that the weakest L…
We adress the problem of spherical deconvolution in a non parametric statistical framework, where both the signal and the operator kernel are subject to error measurements. After a preliminary treatment of the kernel, we apply a thresholding procedure to the signal in a second generation wavelet basis. Under standard assumptions on the kernel, we study the theoritical performance of the resulting algorithm in terms of $L^p$ losses ($p\geq 1$) on Besov spaces on the sphere. We hereby extend the application of second generation spherical wavelets to the blind deconvolution framework. The procedu…
We introduce an original method of multidimensional ridge penalization in functional local linear regressions. The nonparametric regression of functional data is extended from its multivariate counterpart, and is known to be sensitive to the choice of $J$, where $J$ is the dimension of the projection subspace of the data. Under multivariate setting, a roughness penalty is helpful for variance reduction. However, among the limited works covering roughness penalty under the functional setting, most only use a single scalar for tuning. Our new approach proposes a class of data-adaptive ridge pena…
We adress the problem of Laplace deconvolution with random noise in a regression framework. The time set is not considered to be fixed, but grows with the number of observation points. Moreover, the convolution kernel is unknown, and accessible only through experimental noise. We make use of a recent procedure of estimation which couples a Galerkin projection of the operator on Laguerre functions, with a threshold performed both on the operator and the observed signal. We establish the minimax optimality of our procedure under the squared loss error, when the smoothness of the signal is measur…
Quantile regression has become a valuable tool to analyze heterogeneous covaraite-response associations that are often encountered in practice. The development of quantile regression methodology for high-dimensional covariates primarily focuses on examination of model sparsity at a single or multiple quantile levels, which are typically pre-specified ad hoc by the users. The resulting models may be sensitive to the specific choices of the quantile levels, leading to difficulties in interpretation and erosion of confidence in the results. In this article, we propose a new penalization framework…
We study the estimation of the covariance matrix $Σ$ of a $p$-dimensional normal random vector based on $n$ independent observations corrupted by additive noise. Only a general nonparametric assumption is imposed on the distribution of the noise without any sparsity constraint on its covariance matrix. In this high-dimensional semiparametric deconvolution problem, we propose spectral thresholding estimators that are adaptive to the sparsity of $Σ$. We establish an oracle inequality for these estimators under model miss-specification and derive non-asymptotic minimax convergence rates that are …
We consider estimation of the common probability density $f$ of i.i.d. random variables $X_i$ that are observed with an additive i.i.d. noise. We assume that the unknown density $f$ belongs to a class $\mathcal{A}$ of densities whose characteristic function is described by the exponent $\exp(-α|u|^r)$ as $|u|\to \infty$, where $α>0$, $r>0$. The noise density is supposed to be known and such that its characteristic function decays as $\exp(-β|u|^s)$, as $|u| \to \infty$, where $β>0$, $s>0$. Assuming that $r<s$, we suggest a kernel type estimator that is optimal in sharp asymptotical minimax sen…
In the multidimensional setting, we consider the errors-in-variables model. We aim at estimating the unknown nonparametric multivariate regression function with errors in the covariates. We devise an adaptive estimator based on projection kernels on wavelets and a deconvolution operator. We propose an automatic and fully data driven procedure to select the wavelet level resolution. We obtain an oracle inequality and optimal rates of convergence over anisotropic H{ö}lder classes. Our theoretical results are illustrated by some simulations.
We study the reknown deconvolution problem of recovering a distribution function from independent replicates (signal) additively contaminated with random errors (noise), whose distribution is known. We investigate whether a Bayesian nonparametric approach for modelling the latent distribution of the signal can yield inferences with asymptotic frequentist validity under the $L^1$-Wasserstein metric. When the error density is ordinary smooth, we develop two inversion inequalities relating either the $L^1$ or the $L^1$-Wasserstein distance between two mixture densities (of the observations) to th…
In the present paper we consider the problem of estimating a periodic $(r+1)$-dimensional function $f$ based on observations from its noisy convolution. We construct a wavelet estimator of $f$, derive minimax lower bounds for the $L^2$-risk when $f$ belongs to a Besov ball of mixed smoothness and demonstrate that the wavelet estimator is adaptive and asymptotically near-optimal within a logarithmic factor, in a wide range of Besov balls. We prove in particular that choosing this type of mixed smoothness leads to rates of convergence which are free of the "curse of dimensionality" and, hence, a…
The problem of adaptive noisy clustering is investigated. Given a set of noisy observations $Z_i=X_i+ε_i$, $i=1,...,n$, the goal is to design clusters associated with the law of $X_i$'s, with unknown density $f$ with respect to the Lebesgue measure. Since we observe a corrupted sample, a direct approach as the popular {\it $k$-means} is not suitable in this case. In this paper, we propose a noisy $k$-means minimization, which is based on the $k$-means loss function and a deconvolution estimator of the density $f$. In particular, this approach suffers from the dependence on a bandwidth involved…
We study the nonparametric covariance estimation of a stationary Gaussian field X observed on a regular lattice. In the time series setting, some procedures like AIC are proved to achieve optimal model selection among autoregressive models. However, there exists no such equivalent results of adaptivity in a spatial setting. By considering collections of Gaussian Markov random fields (GMRF) as approximation sets for the distribution of X, we introduce a novel model selection procedure for spatial fields. For all neighborhoods m in a given collection M, this procedure first amounts to computing …
We study the following model of hidden Markov chain: $Y_i=X_i+ε_i$, $ i=1,...,n+1$ with $(X_i)$ a real-valued positive recurrent and stationary Markov chain and $(ε_i)_{1\leq i\leq n+1}$ a noise independent of the sequence $(X_i)$ having a known distribution. We present an adaptive estimator of the transition density based on the quotient of a deconvolution estimator of the density of $X_i$ and an estimator of the density of $(X_i,X_{i+1})$. These estimators are obtained by contrast minimization and model selection. We evaluate the $L2$ risk and its rate of convergence for ordinary smooth and …
We consider the model $Z_i=X_i+\varepsilon_i$, for i.i.d. $X_i$'s and $\varepsilon_i$'s and independent sequences $(X_i)_{i\in{\mathbb{N}}}$ and $(\varepsilon_i)_{i\in{\mathbb{N}}}$. The density $f_{\varepsilon}$ of $\varepsilon_1$ is assumed to be known, whereas the one of $X_1$, denoted by $g$, is unknown. Our aim is to estimate linear functionals of $g$, $<ψ,g>$ for a known function $ψ$. We propose a general estimator of $<ψ,g>$ and study the rate of convergence of its quadratic risk as a function of the smoothness of $g$, $f_{\varepsilon}$ and $ψ$. Different contexts with dependent data, s…
A classical condition for fast learning rates is the margin condition, first introduced by Mammen and Tsybakov. We tackle in this paper the problem of adaptivity to this condition in the context of model selection, in a general learning framework. Actually, we consider a weaker version of this condition that allows one to take into account that learning within a small model can be much easier than within a large one. Requiring this "strong margin adaptivity" makes the model selection problem more challenging. We first prove, in a general framework, that some penalization procedures (including …
We focus on the problem of manifold estimation: given a set of observations sampled close to some unknown submanifold $M$, one wants to recover information about the geometry of $M$. Minimax estimators which have been proposed so far all depend crucially on the a priori knowledge of some parameters quantifying the underlying distribution generating the sample (such as bounds on its density), whereas those quantities will be unknown in practice. Our contribution to the matter is twofold: first, we introduce a one-parameter family of manifold estimators $(\hat{M}_t)_{t\geq 0}$ based on a localiz…
This paper studies sparse density estimation via $\ell_1$ penalization (SPADES). We focus on estimation in high-dimensional mixture models and nonparametric adaptive density estimation. We show, respectively, that SPADES can recover, with high probability, the unknown components of a mixture of probability densities and that it yields minimax adaptive density estimates. These results are based on a general sparsity oracle inequality that the SPADES estimates satisfy. We offer a data driven method for the choice of the tuning parameter used in the construction of SPADES. The method uses the gen…
