Bernstein estimators are well-known to avoid the boundary bias problem of traditional kernel estimators. The theoretical properties of these estimators have been studied extensively on compact intervals and hypercubes, but never on the simplex, except for the mean squared error of the density estimator in Tenbusch (1994) when $d = 2$. The simplex is an important case as it is the natural domain of compositional data. In this paper, we make an effort to prove several asymptotic results (bias, variance, mean squared error (MSE), mean integrated squared error (MISE), asymptotic normality, uniform strong consistency) for Bernstein estimators of cumulative distribution functions and density functions on the $d$-dimensional simplex. Our results generalize the ones in Leblanc (2012) and Babu et al. (2002), who treated the case $d = 1$, and significantly extend those found in Tenbusch (1994). In particular, our rates of convergence for the MSE and MISE are optimal.
In this paper, we study the asymptotic properties (bias, variance, mean squared error) of Bernstein estimators for cumulative distribution functions and density functions near and on the boundary of the $d$-dimensional simplex. Our results generalize those found by Leblanc (2012), who treated the case $d=1$, and complement the results from Ouimet (2021) in the interior of the simplex. Since the "edges" of the $d$-dimensional simplex have dimensions going from $0$ (vertices) up to $d - 1$ (facets) and our kernel function is multinomial, the asymptotic expressions for the bias, variance and mean…
We study theoretically, for the first time, the Dirichlet kernel estimator introduced by Aitchison and Lauder (1985) for the estimation of multivariate densities supported on the $d$-dimensional simplex. The simplex is an important case as it is the natural domain of compositional data and has been neglected in the literature on asymmetric kernels. The Dirichlet kernel estimator, which generalizes the (non-modified) unidimensional Beta kernel estimator from Chen (1999), is free of boundary bias and non-negative everywhere on the simplex. We show that it achieves the optimal convergence rate $O…
Inference on an extreme-value copula usually proceeds via its Pickands dependence function, which is a convex function on the unit simplex satisfying certain inequality constraints. In the setting of an iid random sample from a multivariate distribution with known margins and unknown extreme-value copula, an extension of the Capéraà-Fougères-Genest estimator was introduced by D. Zhang, M. T. Wells and L. Peng [Journal of Multivariate Analysis 99 (2008) 577-588]. The joint asymptotic distribution of the estimator as a random function on the simplex was not provided. Moreover, implementation of …
On the Dirichlet-kernel Gasser--Müller estimator and its competitors for fixed design regression on the simplex
Université de Tunis El-Manar（突尼斯埃尔马纳大学）；McGill University（麦吉尔大学）；Univ Rennes, CNRS, IRMAR - UMR 6625（雷恩大学，法国国家科学研究中心，IRMAR - UMR 6625）；Université du Québec à Trois-Rivières（魁北克省特鲁瓦里维埃尔斯大学）
A discrete statistical model is a subset of a probability simplex. Its maximum likelihood estimator (MLE) is a retraction from that simplex onto the model. We characterize all models for which this retraction is a rational function. This is a contribution via real algebraic geometry which rests on results due to Huh and Kapranov on Horn uniformization. We present an algorithm for constructing models with rational MLE, and we demonstrate it on a range of instances. Our focus lies on models familiar to statisticians, like Bayesian networks, decomposable graphical models, and staged trees.
We prove the asymptotic normality of the kernel density estimator (introduced by Rosenblatt (1956) and Parzen (1962)) in the context of stationary strongly mixing random fields. Our approach is based on the Lindeberg's method rather than on Bernstein's small-block-large-block technique and coupling arguments widely used in previous works on nonparametric estimation for spatial processes. Our method allows us to consider only minimal conditions on the bandwidth parameter and provides a simple criterion on the (non-uniform) strong mixing coefficients which do not depend on the bandwith.
This paper brings a contribution to the Bayesian theory of nonparametric and semiparametric estimation. We are interested in the asymptotic normality of the posterior distribution in Gaussian linear regression models when the number of regressors increases with the sample size. Two kinds of Bernstein-von Mises Theorems are obtained in this framework: nonparametric theorems for the parameter itself, and semiparametric theorems for functionals of the parameter. We apply them to the Gaussian sequence model and to the regression of functions in Sobolev and $C^α$ classes, in which we get the minima…
Shape restrictions on functional regression coefficients such as non-negativity, monotonicity, convexity or concavity are often available in the form of a prior knowledge or required to maintain a structural consistency in functional regression models. A new estimation method is developed in shape-constrained functional regression models using Bernstein polynomials. Specifically, estimation approaches from nonparametric regression are extended to functional data, properly accounting for shape-constraints in a large class of functional regression models such as scalar-on-function regression (SO…
Dirichlet kernel density estimation on the simplex with missing data
Imam Mohammad Ibn Saud Islamic University（伊玛目穆罕默德·本·沙特伊斯兰大学）；Université de Tunis El-Manar（突尼斯埃尔曼纳尔大学）；Dalhousie University（达尔豪斯大学）；Université du Québec à Trois-Rivières（魁北克特罗瓦里维尔斯大学）
The prominent Bernstein -- von Mises (BvM) result claims that the posterior distribution after centering by the efficient estimator and standardizing by the square root of the total Fisher information is nearly standard normal. In particular, the prior completely washes out from the asymptotic posterior distribution. This fact is fundamental and justifies the Bayes approach from the frequentist viewpoint. In the nonparametric setup the situation changes dramatically and the impact of prior becomes essential even for the contraction of the posterior; see [vdV2008], [Bo2011], [CaNi2013,CaNi2014]…
We study Empirical Risk Minimizers (ERM) and Regularized Empirical Risk Minimizers (RERM) for regression problems with convex and $L$-Lipschitz loss functions. We consider a setting where $|\cO|$ malicious outliers contaminate the labels. In that case, under a local Bernstein condition, we show that the $L_2$-error rate is bounded by $ r_N + AL |\cO|/N$, where $N$ is the total number of observations, $r_N$ is the $L_2$-error rate in the non-contaminated setting and $A$ is a parameter coming from the local Bernstein condition. When $r_N$ is minimax-rate-optimal in a non-contaminated setting, th…
Many applications in risk analysis, especially in environmental sciences, require the estimation of the dependence among multivariate maxima. A way to do this is by inferring the Pickands dependence function of the underlying extreme-value copula. A nonparametric estimator is constructed as the sample equivalent of a multivariate extension of the madogram. Shape constraints on the family of Pickands dependence functions are taken into account by means of a representation in terms of a specific type of Bernstein polynomials. The large-sample theory of the estimator is developed and its finite-s…
Maximum likelihood estimators are proposed for the parameters and the densities in a semiparametric density ratio model in which the nonparametric baseline density is approximated by the Bernstein polynomial model. The EM algorithm is used to obtain the maximum approximate Bernstein likelihood estimates. Simulation study shows that the performance of the proposed method is much better than the existing ones. The proposed method is illustrated by real data examples. Some asymptotic results are also presented and proved.
A new maximum likelihood method for deconvoluting a continuous density with a positive lower bound on a known compact support in additive measurement error models with known error distribution using the approximate Bernstein type polynomial model, a finite mixture of specific beta distributions, is proposed. The change-point detection method is used to choose an optimal model degree. Based on a contaminated sample of size $n$, under an assumption which is satisfied, among others, by the generalized normal error distribution, the optimal rate of convergence of the mean integrated squared error …
A broad class of smooth, possibly data-adaptive nonparametric copula estimators that contains empirical Bernstein copulas introduced by Sancetta and Satchell (and thus the empirical beta copula proposed by Segers, Sibuya and Tsukahara) is studied. Within this class, a subclass of estimators that depend on a scalar parameter determining the amount of marginal smoothing and a functional parameter controlling the shape of the smoothing region is specifically considered. Empirical investigations of the influence of these parameters suggest to focus on two particular data-adaptive smooth copula est…
In a smooth semiparametric estimation problem, the marginal posterior for the parameter of interest is expected to be asymptotically normal and satisfy frequentist criteria of optimality if the model is endowed with a suitable prior. It is shown that, under certain straightforward and interpretable conditions, the assertion of Le Cam's acclaimed, but strictly parametric, Bernstein-von Mises theorem [Univ. California Publ. Statist. 1 (1953) 277-329] holds in the semiparametric situation as well. As a consequence, Bayesian point-estimators achieve efficiency, for example, in the sense of Hájek's…
The main goal of this paper is to study the parameter estimation problem, using the Bayesian methodology, for the drift coefficient of some linear (parabolic) SPDEs driven by a multiplicative noise of special structure. We take the spectral approach by assuming that one path of the first $N$ Fourier modes of the solution is continuously observed over a finite time interval. First, we show that the model is regular and fits into classical local asymptotic normality framework, and thus the MLE and the Bayesian estimators are weakly consistent, asymptotically normal, efficient, and asymptotically…
In this paper, we consider the alleviation of the boundary problem when the probability density function has bounded support. We apply Robbins-Monro's algorithm and Bernstein polynomials to construct a recursive density estimator. We study the asymptotic properties of the proposed recursive estimator. We then compared our proposed recursive estimator with many others estimators. Finally, we confirm our theoretical result through a simulation study and then using two real datasets.
In this work, we establish the asymptotic normality of the deconvolution kernel density estimator in the context of strongly mixing random fields. Only minimal conditions on the bandwidth parameter are required and a simple criterion on the strong mixing coefficients is provided. Our approach is based on the Lindeberg's method rather than on Bernstein's technique and coupling arguments widely used in previous works on nonparametric estimation for spatial processes. We deal also with nonmixing random fields which can be written as a (nonlinear) functional of i.i.d. random fields by considering …
In the field of finance, insurance, and system reliability, etc., it is often of interest to measure the dependence among variables by modeling a multivariate distribution using a copula. The copula models with parametric assumptions are easy to estimate but can be highly biased when such assumptions are false, while the empirical copulas are non-smooth and often not genuine copula making the inference about dependence challenging in practice. As a compromise, the empirical Bernstein copula provides a smooth estimator but the estimation of tuning parameters remains elusive. In this paper, by u…
We study Voronoi cells in the statistical setting by considering preimages of the maximum likelihood estimator that tessellate an open probability simplex. In general, logarithmic Voronoi cells are convex sets. However, for certain algebraic models, namely finite models, models with ML degree 1, linear models, and log-linear (or toric) models, we show that logarithmic Voronoi cells are polytopes. As a corollary, the algebraic moment map has polytopes for both its fibres and its image, when restricted to the simplex. We also compute non-polytopal logarithmic Voronoi cells using numerical algebr…
We consider a system of $N$ interacting particles, governed by transport and diffusion, that converges in a mean-field limit to the solution of a McKean-Vlasov equation. From the observation of a trajectory of the system over a fixed time horizon, we investigate nonparametric estimation of the solution of the associated nonlinear Fokker-Planck equation, together with the drift term that controls the interactions, in a large population limit $N \rightarrow \infty$. We build data-driven kernel estimators and establish oracle inequalities, following Lepski's principle. Our results are based on a …
We investigate the validity of two resampling techniques when carrying out inference on the underlying unknown copula using a recently proposed class of smooth, possibly data-adaptive nonparametric estimators that contains empirical Bernstein copulas (and thus the empirical beta copula). Following \cite{KirSegTsu21}, the first resampling technique is based on drawing samples from the smooth estimator and can only can be used in the case of independent observations. The second technique is a smooth extension of the so-called sequential dependent multiplier bootstrap and can thus be used in a ti…
Université Jean Monnet, ICJ UMR5208, CNRS, Ecole Centrale de Lyon, INSA Lyon, Universite Claude Bernard Lyon 1, Saint-Étienne, France（法国圣艾蒂安大学，ICJ UMR5208，CNRS，中央理工 Lyon 学院，INSA Lyon，Université Claude Bernard Lyon 1）
We present Hoeffding-type and Bernstein-type inequalities for right-censored data. The inequalities bound the difference between an inverse of the probability of censoring weighting (IPCW) estimator and its expectation. We first discuss the asymptotic properties of the estimator and provide conditions for its efficiency. We present standard, data dependent, and uniform Hoeffding-type inequalities. We then present a Bernstein-type inequality. Finally, we show how to apply these inequalities in an empirical risk minimization setting.
Extreme-value copulas arise in the asymptotic theory for componentwise maxima of independent random samples. An extreme-value copula is determined by its Pickands dependence function, which is a function on the unit simplex subject to certain shape constraints that arise from an integral transform of an underlying measure called spectral measure. Multivariate extensions are provided of certain rank-based nonparametric estimators of the Pickands dependence function. The shape constraint that the estimator should itself be a Pickands dependence function is enforced by replacing an initial estima…
The Negative Binomial distribution becomes highly skewed under extreme dispersion. Even at moderately large sample sizes, the sample mean exhibits a heavy right tail. The standard Normal approximation often does not provide adequate inferences about the data's mean in this setting. In previous work, we have examined alternative methods of generating confidence intervals for the expected value. These methods were based upon Gamma and Chi Square approximations or tail probability bounds such as Bernstein's Inequality. We now propose growth estimators of the Negative Binomial mean. Under high dis…
Simultaneous inference for high-dimensional non-Gaussian time series is always considered to be a challenging problem. Such tasks require not only robust estimation of the coefficients in the random process, but also deriving limiting distribution for a sum of dependent variables. In this paper, we propose a multiplier bootstrap procedure to conduct simultaneous inference for the transition coefficients in high-dimensional non-Gaussian vector autoregressive (VAR) models. This bootstrap-assisted procedure allows the dimension of the time series to grow exponentially fast in the number of observ…
The ordinary Bayes estimator based on the posterior density suffers from the potential problems of non-robustness under data contamination or outliers. In this paper, we consider the general set-up of independent but non-homogeneous (INH) observations and study a robustified pseudo-posterior based estimation for such parametric INH models. In particular, we focus on the $R^{(α)}$-posterior developed by Ghosh and Basu (2016) for IID data and later extended by Ghosh and Basu (2017) for INH set-up, where its usefulness and desirable properties have been numerically illustrated. In this paper, we …
We study large sample properties of Bayesian analysis of the proportional hazard model with neutral to the right process priors on the baseline hazard function. We show that the posterior distribution of the baseline cumulative hazard function and regression coefficients centered at the maximum likelihood estimator is jointly asymptotically equivalent to the sampling distribution of the maximum likelihood estimator.
This article presents an important theorem, which shows that from the moments of the standard normal distribution one can generate density functions originating a family of models. Additionally, we discussed that different random variable domains are achieved with transformations. For instance, we adopted the moment of order two, from the proposed theorem, and transformed it, which allowed us to exemplify this class as unit distribution. We named it as Alpha-Unit (AU) distribution, which contains a single positive parameter $α$ ($\text{AU}(α) \in [0,1]$). We presented its properties and showed…
In a smooth semi-parametric model, the marginal posterior distribution for a finite dimensional parameter of interest is expected to be asymptotically equivalent to the sampling distribution of any efficient point-estimator. The assertion leads to asymptotic equivalence of credible and confidence sets for the parameter of interest and is known as the semi-parametric Bernstein-von Mises theorem. In recent years, it has received much attention and has been applied in many examples. We consider models in which errors with symmetric densities play a role; more specifically, it is shown that the ma…
In a smooth semiparametric model, the marginal posterior distribution of the finite dimensional parameter of interest is expected to be asymptotically equivalent to the sampling distribution of frequentist's efficient estimators. This is the assertion of the so-called Bernstein-von Mises theorem, and recently, it has been proved in many interesting semiparametric models. In this thesis, we consider the semiparametric Bernstein-von Mises theorem in some models which have symmetric errors. The simplest example of these models is the symmetric location model that has 1-dimensional location parame…
The local Rademacher complexity framework is one of the most successful general-purpose toolboxes for establishing sharp excess risk bounds for statistical estimators based on the framework of empirical risk minimization. Applying this toolbox typically requires using the Bernstein condition, which often restricts applicability to convex and proper settings. Recent years have witnessed several examples of problems where optimal statistical performance is only achievable via non-convex and improper estimators originating from aggregation theory, including the fundamental problem of model select…
Adaptive deep nonparametric regression from dependent data under covariate shift
Université Jean Monnet, ICJ UMR5208, CNRS, Ecole Centrale de Lyon, INSA Lyon, Universite Claude Bernard Lyon 1（里昂 Jean Monnet 大学，ICJ UMR5208，国家科学研究中心，里昂中央理工大学，里昂国家理工学校，里昂大学 Claude Bernard 1）
In this paper, we prove a local limit theorem for the ratio of the Poisson distribution to the Gaussian distribution with the same mean and variance, using only elementary methods (Taylor expansions and Stirling's formula). We then apply the result to derive an upper bound on the Le Cam distance between Poisson and Gaussian experiments, which gives a complete proof of the sketch provided in the unpublished set of lecture notes by Pollard (2010), who uses a different approach. We also use the local limit theorem to derive the asymptotics of the variance for Bernstein c.d.f. and density estimato…
