In this paper we prove large and moderate deviations principles for the recursive kernel estimator of a probability density function and its partial derivatives. Unlike the density estimator, the derivatives estimators exhibit a quadratic behavior not only for the moderate deviations scale but also for the large deviations one. We provide results both for the pointwise and the uniform deviations.
In this paper we prove large and moderate deviations principles for the recursive kernel estimators of a probability density function defined by the stochastic approximation algorithm introduced by Mokkadem et al. [2009. The stochastic approximation method for the estimation of a probability density. J. Statist. Plann. Inference 139, 2459-2478]. We show that the estimator constructed using the stepsize which minimize the variance of the class of the recursive estimators defined in Mokkadem et al. (2009) gives the same pointwise LDP and MDP as the Rosenblatt kernel estimator. We provide results…
We apply the stochastic approximation method to construct a large class of recursive kernel estimators of a probability density, including the one introduced by Hall and Patil (1994). We study the properties of these estimators and compare them with Rosenblatt's nonrecursive estimator. It turns out that, for pointwise estimation, it is preferable to use the nonrecursive Rosenblatt's kernel estimator rather than any recursive estimator. A contrario, for estimation by confidence intervals, it is better to use a recursive estimator rather than Rosenblatt's estimator.
In this paper, we prove large deviations principle for the Nadaraya-Watson estimator and for the semi-recursive kernel estimator of the regression in the multidimensional case. Under suitable conditions, we show that the rate function is a good rate function. We thus generalize the results already obtained in the unidimensional case for the Nadaraya-Watson estimator. Moreover, we give a moderate deviations principle for these two estimators. It turns out that the rate function obtained in the moderate deviations principle for the semi-recursive estimator is larger than the one obtained for the…
We derive the exponential non improvable Grand Lebesgue Space norm decreasing estimations for tail of distribution for exact normed deviation for the famous recursive Wolverton-Wagner multivariate statistical density estimation. We consider pointwise as well as Lebesgue-Riesz norm error of statistical density of measurement.
The paper deals with the statistical analysis of several data sets associated with shape invariant models with different translation, height and scaling parameters. We propose to estimate these parameters together with the common shape function. Our approach extends the recent work of Bercu and Fraysse to multivariate shape invariant models. We propose a very efficient Robbins-Monro procedure for the estimation of the translation parameters and we use these estimates in order to evaluate scale parameters. The main pattern is estimated by a weighted Nadaraya-Watson recursive estimator. We provi…
This paper describes a recursive estimation procedure for multivariate binary densities (probability distributions of vectors of Bernoulli random variables) using orthogonal expansions. For $d$ covariates, there are $2^d$ basis coefficients to estimate, which renders conventional approaches computationally prohibitive when $d$ is large. However, for a wide class of densities that satisfy a certain sparsity condition, our estimator runs in probabilistic polynomial time and adapts to the unknown sparsity of the underlying density in two key ways: (1) it attains near-minimax mean-squared error fo…
We introduce a nonparametric prior on the conditional distribution of a (univariate or multivariate) response given a set of predictors. The prior is constructed in the form of a two-stage generative procedure, which in the first stage recursively partitions the predictor space, and then in the second stage generates the conditional distribution by a multi-scale nonparametric density model on each predictor partition block generated in the first stage. This design allows adaptive smoothing on both the predictor space and the response space, and it results in the full posterior conjugacy of the…
Non-linear aggregation strategies have recently been proposed in response to the problem of how to combine, in a non-linear way, estimators of the regression function (see for instance \cite{biau:16}), classification rules (see \cite{ch:16}), among others. Although there are several linear strategies to aggregate density estimators, most of them are hard to compute (even in moderate dimensions). Our approach aims to overcome this problem by estimating the density at a point $x$ using not just sample points close to $x$ but in a neighborhood of the (estimated) level set $f(x)$. We show, both th…
In this paper we prove large deviations principles for the averaged stochastic approximation method for the estimation of a regression function introduced by A. Mokkadem et al. [Revisiting Révész's stochastic approximation method for the estimation of a regression function, ALEA Lat. Amm. J. Probab. Math. Stat. 6 (2009), 63-114]. We show that the averaged stochastic approximation algorithm constructed using the weight sequence which minimize the asymptotic variance gives the same pointwise LDP as the Nadaraya-Watson kernel estimator. Moreover, we give a moderate deviations principle for these …
In this paper, we propose a test for the equality of multiple distributions based on kernel mean embeddings. Our framework provides a flexible way to handle multivariate or even high-dimensional data by virtue of kernel methods and allows the number of distributions to increase with the sample size. This is in contrast to previous studies that have been mostly restricted to classical low-dimensional settings with a fixed number of distributions. By building on Cramer-type moderate deviation for degenerate two-sample V-statistics, we derive the limiting null distribution of the test statistic a…
We consider a system of dependent Poisson variables, where each variable is the sum of an independent variate and a common variate. It is the common variate that creates the dependence. Within this system, a test of independence may be constructed where the null hypothesis is that the common variate is identically zero. In the present paper, we consider the maximum log likelihood ratio test. For this test, it is well-known that the asymptotic distribution of the test statistic is an equal mixture of zero and a chi square distribution with one degree of freedom. We examine a Bartlett correction…
Conditional independence, graphical models and sparsity are key notions for parsimonious statistical models and for understanding the structural relationships in the data. The theory of multivariate and spatial extremes describes the risk of rare events through asymptotically justified limit models such as max-stable and multivariate Pareto distributions. Statistical modelling in this field has been limited to moderate dimensions so far, partly owing to complicated likelihoods and a lack of understanding of the underlying probabilistic structures. We introduce a general theory of conditional i…
Full likelihood-based inference for high-dimensional multivariate extreme value distributions, or max-stable processes, is feasible when incorporating occurrence times of the maxima; without this information, $d$-dimensional likelihood inference is usually precluded due to the large number of terms in the likelihood. However, some studies have noted bias when performing high-dimensional inference that incorporates such event information, particularly when dependence is weak. We elucidate this phenomenon, showing that for unbiased inference in moderate dimensions, dimension $d$ should be of a m…
In this paper, we consider the problem of estimating a conditional density in moderately large dimensions. Much more informative than regression functions, conditional densities are of main interest in recent methods, particularly in the Bayesian framework (studying the posterior distribution, finding its modes...). Considering a recently studied family of kernel estimators, we select a pointwise multivariate bandwidth by revisiting the greedy algorithm Rodeo (Regularisation Of Derivative Expectation Operator). The method addresses several issues: being greedy and computationally efficient by …
Many developments in Mathematics involve the computation of higher order derivatives of Gaussian density functions. The analysis of univariate Gaussian random variables is a well-established field whereas the analysis of their multivariate counterparts consists of a body of results which are more dispersed. These latter results generally fall into two main categories: theoretical expressions which reveal the deep structure of the problem, or computational algorithms which can mask the connections with closely related problems. In this paper, we unify existing results and develop new results in…
UC Irvine（加州大学欧文分校）；UC San Diego（加州大学圣迭戈分校）；Microsoft Research New England（微软新英格兰研究院）；Wharton School, University of Pennsylvania（宾夕法尼亚大学沃顿商学院）；Carnegie Mellon University（卡内基梅隆大学）；Rutgers University（罗格斯大学）
This work considers the problem of learning the structure of multivariate linear tree models, which include a variety of directed tree graphical models with continuous, discrete, and mixed latent variables such as linear-Gaussian models, hidden Markov models, Gaussian mixture models, and Markov evolutionary trees. The setting is one where we only have samples from certain observed variables in the tree, and our goal is to estimate the tree structure (i.e., the graph of how the underlying hidden variables are connected to each other and to the observed variables). We propose the Spectral Recurs…
For the analysis of clustered survival data, two different types of models that take the association into account, are commonly used: frailty models and copula models. Frailty models assume that conditional on a frailty term for each cluster, the hazard functions of individuals within that cluster are independent. These unknown frailty terms with their imposed distribution are used to express the association between the different individuals in a cluster. Copula models on the other hand assume that the joint survival function of the individuals within a cluster is given by a copula function, e…
We propose a novel estimator for the number of components (denoted by $M$) in a K-variate non-parametric finite mixture model, where the analyst has repeated observations of $K\geq2$ variables that are independent given a finitely supported unobserved variable. Under a mild assumption on the joint distribution of the observed and latent variables, we show that an integral operator $T$, that is identified from the data, has rank equal to $M$. Using this observation, and the fact that singular values are stable under perturbations, the estimator of $M$ that we propose is based on a thresholding …
We present multivariate unbiased estimators for second, third, and fourth order cumulants $C_2(x,y)$, $C_3(x,y,z)$, and $C_4(x,y,z,w)$. Many relevant new estimators are derived for cases where some variables are average-free or pairs of variables have a vanishing second order cumulant. The well-know Fisher k-statistics is recovered for the single variable case. The variances of several estimators are explicitly given in terms of higher order cumulants and discussed with respect to random processes that are predominately Gaussian. We surprisingly find that the frequently used third order estima…
