Uwe Einmahl, David M. Mason
We introduce a general method to prove uniform in bandwidth consistency of kernel-type function estimators. Examples include the kernel density estimator, the Nadaraya-Watson regression estimator and the conditional empirical process. Our results may be useful to establish uniform consistency of data-driven bandwidth kernel-type function estimators.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
继续探索
In this paper we study the ideal variable bandwidth kernel density estimator introduced by McKay (1993) and Jones, McKay and Hu (1994) and the plug-in practical version of the variable bandwidth kernel estimator with two sequences of bandwidths as in Giné and Sang (2013). Based on the bias and variance analysis of the ideal and true variable bandwidth kernel density estimators, we study the central limit theorems for each of them.
Nonparametric density and regression estimators commonly depend on a bandwidth. The asymptotic properties of these estimators have been widely studied when bandwidths are nonstochastic. In practice, however, in order to improve finite sample performance of these estimators, bandwidths are selected by data driven methods, such as cross-validation or plug-in procedures. As a result nonparametric estimators are usually constructed using stochastic bandwidths. In this paper we establish the asymptotic equivalence in probability of local polynomial regression estimators under stochastic and nonstoc…
We establish uniform-in-bandwidth consistency for kernel-type estimators of the differential entropy. We consider two kernel-type estimators of Shannon's entropy. As a consequence, an asymptotic 100% confidence interval of entropy is provided.
In a regression model, we write the Nadaraya-Watson estimator of the regression function as the quotient of two kernel estimators, and propose a bandwidth selection method for both the numerator and the denominator. We prove risk bounds for both data driven estimators and for the resulting ratio. The simulation study confirms that both estimators have good performances, compared to the ones obtained by cross-validation selection of the bandwidth. However, unexpectedly, the single-bandwidth cross-validation estimator is found to be much better than the ratio of the previous two good estimators,…
In this paper we establish the uniform in bandwidth consistency for the transformation kernel estimator of copulas introduced in [Omelka et al.(2009)]. To this end, we first prove a uniform in bandwidth law of the iterated logarithm for the maximal deviation of this estimator from its expectation. We then show that, as n goes to infinity, the bias of the estimator converges to zero uniformly in the bandwidth h, varying over a suitable interval. A practical method of selecting the optimal bandwidth is also presented. Finally, we make conclusive simulation experiments showing the performance of …
On large bandwidth matrix values kernel smoothed estimators for multi-index models
本研究探讨了多索引模型中大带宽矩阵值核平滑估计量的渐近性质，发现其能自然减少维度灾难，通过数值实验和案例研究验证了其有效性。
For $n$ independent random variables having the same Hölder continuous density, this paper deals with controls of the Wolverton-Wagner's estimator MSE and MISE. Then, for a bandwidth $h_n(β)$, estimators of $β$ are obtained by a Goldenshluger-Lepski type method and a Lacour-Massart-Rivoirard type method. Some numerical experiments are provided for this last method.
In this paper we propose an automatic selection of the bandwidth of the semi-recursive kernel estimators of a regression function defined by the stochastic approximation algorithm. We showed that, using the selected bandwidth and some special stepsizes, the proposed semi-recursive estimators will be very competitive to the nonrecursive one in terms of estimation error but much better in terms of computational costs. We corroborated these theoretical results through simulation study and a real dataset.
It is shown that the Hall, Hu and Marron [Hall, P., Hu, T., and Marron J.S. (1995), Improved Variable Window Kernel Estimates of Probability Densities, {\it Annals of Statistics}, 23, 1--10] modification of Abramson's [Abramson, I. (1982), On Bandwidth Variation in Kernel Estimates - A Square-root Law, {\it Annals of Statistics}, 10, 1217--1223] variable bandwidth kernel density estimator satisfies the optimal asymptotic properties for estimating densities with four uniformly continuous derivatives, uniformly on bounded sets where the preliminary estimator of the density is bounded away from z…
In this paper we consider the kernel estimators of a distribution function defined by the stochastic approximation algorithm when the observation are contamined by measurement errors. It is well known that this estimators depends heavily on the choice of a smoothing parameter called the bandwidth. We propose a specific second generation plug-in method of the deconvolution kernel distribution estimators defined by the stochastic approximation algorithm. We show that, using the proposed bandwidth selection and the stepsize which minimize the MISE (Mean Integrated Squared Error), the proposed est…
A note on kernel density estimators with optimal bandwidths
本文研究了最优带宽核密度估计器的累积分布函数，证明其在样本量增加时几乎必然位于经验分布函数的置信区间外。
We generalize a method for proving uniform in bandwidth consistency results for kernel type estimators developed by the two last named authors. Such results are shown to be useful in establishing consistency of local polynomial estimators of the regression function.
This study investigates the effect of bandwidth selection via a plug-in method on the asymptotic structure of the nonparametric kernel density estimator. We generalise the result of Hall and Kang (2001) and find that the plug-in method has no effect on the asymptotic structure of the estimator up to the order of $O\{(nh_0)^{-1/2}+h_0^L\}=O(n^{-L/(2L+1)})$ for a bandwidth $h_0$ and any kernel order $L$ when the kernel order for pilot estimation $L_p$ is high enough. We also provide the valid Edgeworth expansion up to the order of $O\{(nh_0)^{-1}+h_0^{2L}\}$ and find that, as long as the $L_p$ i…
We are interested in the rate of consistency of kernel density estimators with respect to the weighted sup-norm determined by some unbounded weight function. This problem has been considered by Gine, Koltchinskii and Zinn (2004) for a deterministic bandwidth sequence. We provide "uniform in h" versions of some of their results, allowing us to determine the corresponding rates of consistency for kernel density estimators where the bandwidth sequences may depend on the data and/or the location.
We investigate the asymptotic mean squared error of kernel estimators of the intensity function of a spatial point process. We show that when $n$ independent copies of a point process in $\mathbb R^d$ are superposed, the optimal bandwidth $h_n$ is of the order $n^{-1/(d+4)}$ under appropriate smoothness conditions on the kernel and true intensity function. We apply the Abramson principle to define adaptive kernel estimators and show that asymptotically the optimal adaptive bandwidth is of the order $n^{-1/(d+8)}$ under appropriate smoothness conditions.
We consider the derivation of data-dependent simultaneous bandwidths for double kernel heteroskedasticity and autocorrelation consistent (DK-HAC) estimators. In addition to the usual smoothing over lagged autocovariances for classical HAC estimators, the DK-HAC estimator also applies smoothing over the time direction. We obtain the optimal bandwidths that jointly minimize the global asymptotic MSE criterion and discuss the trade-off between bias and variance with respect to smoothing over lagged autocovariances and over time. Unlike the MSE results of Andrews (1991), we establish how nonstatio…
In a Bayesian framework we prove that the optimal estimator of a conditional density is consistent.
We describe a four-level hierarchy mapping both all discrete estimation problems and all estimators on these problems, such that the hierarchy describes each estimator's consistency guarantees on each problem class. We show that no estimator is consistent for all estimation problems, but that some estimators, such as Maximum A Posteriori, are consistent for the widest possible class of discrete estimation problems. For Maximum Likelihood and Approximate Maximum Likelihood estimators we show that they do not provide consistency on as wide a class, but define a sub-class of problems characterise…
An effective estimation of multivariate density functions using extended-beta kernels with Bayesian adaptive bandwidths
针对多元密度函数估计问题，提出带贝叶斯自适应带宽的多重扩展β核（MEBK）估计器，建立其渐近性质并提供带宽选择方法，经模拟与真实数据验证，在灵活性和通用性上优于高斯、伽马核估计器。
Strong consistency of the Bayes estimator of a regression curve for the $L^1$-squared loss function is proved. It is also shown the convergence to 0 of the Bayes risk of this estimator both for the $L^1$ and $L^1$-squared loss functions. The Bayes estimator of a regression curve is the regression curve with respect to the posterior predictive distribution.
Rates of strong uniform consistency for the $k$-nearest neighbors kernel estimators of density and regression function
本文探讨了多元情形下密度与回归函数的 $k$-最近邻核估计量的一致性问题，并在特定假设下推导出这些估计量在整个空间 $\mathbb{R}^p$ 上的强一致一致性的收敛速率。
In this article, basing on NQD samples, we investigate the fixed design nonparametric regression model, where the errors are pairwise NQD random errors, with fixed design points, and an unknown function. Nonparametric weighted estimator will be introduced and its consistency is studied. As special case, the consistency result for weighted kernel estimators of the model is obtained. This extends the earlier work on independent random and dependent random errors to NQD case.
Under mild conditions, it is shown the strong consistency of the Bayes estimator of the density. Moreover, the Bayes risk (for some common loss functions) of the Bayes estimator of the density (i.e. the posterior predictive density) reaches zero when the sample size goes to $\infty$. In passing, a similar result is obtained for the estimation of the sampling distribution.
Given an i.i.d sample $(Y_i,Z_i)$, taking values in $\RRR^{d'}\times \RRR^d$, we consider a collection Nadarya-Watson kernel estimators of the conditional expectations $\EEE(<c_g(z),g(Y)>+d_g(z)\mid Z=z)$, where $z$ belongs to a compact set $H\subset \RRR^d$, $g$ a Borel function on $\RRR^{d'}$ and $c_g(\cdot),d_g(\cdot)$ are continuous functions on $\RRR^d$. Given two bandwidth sequences $h_n<\wth_n$ fulfilling mild conditions, we obtain an exact and explicit almost sure limit bounds for the deviations of these estimators around their expectations, uniformly in $g\in\GG,\;z\in H$ and $h_n\le …
In this article we perform an asymptotic analysis of Bayesian parallel kernel density estimators introduced by Neiswanger, Wang and Xing (2014). We derive the asymptotic expansion of the mean integrated squared error for the full data posterior estimator and investigate the properties of asymptotically optimal bandwidth parameters. Our analysis demonstrates that partitioning data into subsets requires a non-trivial choice of bandwidth parameters that optimizes the estimation error.
The author establishes a new mathematical expression for the Frequency Polygon. He uses it to prove the strong uniform consistency of the Frequency Polygon marginal density estimator for non-anticipative stationary stochastic processes which are stable in the sense of Wu. He gives examples of several times series models for which this result is relevant.
This paper deals with a general class of observation-driven time series models with a special focus on time series of counts. We provide conditions under which there exist strict-sense stationary and ergodic versions of such processes. The consistency of the maximum likelihood estimators is then derived for well- specified and misspecified models.
This article develops a unified framework to study the asymptotic properties of all periodic spline-based estimators, that is, of regression, penalized and smoothing splines. The explicit form of the periodic Demmler-Reinsch basis in terms of exponential splines allows the derivation of an expression for the asymptotic equivalent kernel on the real line for all spline estimators simultaneously. The corresponding bandwidth, which drives the asymptotic behavior of spline estimators, is shown to be a function of the number of knots and the smoothing parameter. Strategies for the selection of the …
The Kaplan-Meier estimator as a limit function of Efron's self-consistency iterations
本文针对Efron提出的自洽迭代算法，证明其可逼近Kaplan--Meier估计量这一不动点的收敛性，明确该估计量是迭代的极限函数。
A Proof of Strong Consistency of Maximum Likelihood Estimator for Independent Non-Identically Distributed Data
该研究在密度函数满足特定假设的条件下，基于已有研究给出独立非同分布数据下极大似然估计量强相合性的通用证明，并将其用于证明轨道确定问题中极大似然估计量的强相合性。
The stochastic block model (SBM) is a probabilistic model de- signed to describe heterogeneous directed and undirected graphs. In this paper, we address the asymptotic inference on SBM by use of maximum- likelihood and variational approaches. The identi ability of SBM is proved, while asymptotic properties of maximum-likelihood and variational esti- mators are provided. In particular, the consistency of these estimators is settled, which is, to the best of our knowledge, the rst result of this type for variational estimators with random graphs.
In this paper we propose an automatic bandwidth selection of the recursive kernel density estimators with missing data in the context of global and local density estimation. We showed that, using the selected bandwidth and a special stepsize, the proposed recursive estimators outperformed the nonrecursive one in terms of estimation error in the case of global estimation. However, the recursive estimators are much better in terms of computational costs. We corroborated these theoretical results through simulation studies and on the simulated data of the Aquitaine cohort of HIV-1 infected patien…
Markov chain Monte Carlo (MCMC) algorithms are used to estimate features of interest of a distribution. The Monte Carlo error in estimation has an asymptotic normal distribution whose multivariate nature has so far been ignored in the MCMC community. We present a class of multivariate spectral variance estimators for the asymptotic covariance matrix in the Markov chain central limit theorem and provide conditions for strong consistency. We examine the finite sample properties of the multivariate spectral variance estimators and its eigenvalues in the context of a vector autoregressive process …
We consider the high order moments estimator of the frontier of a random pair introduced by Girard, S., Guillou, A., Stupfler, G. (2012). {\it Frontier estimation with kernel regression on high order moments}. In the present paper, we show that this estimator is strongly uniformly consistent on compact sets and its rate of convergence is given when the conditional cumulative distribution function belongs to the Hall class of distribution functions.
Higher-order Refinements of Small Bandwidth Asymptotics for Density-Weighted Average Derivative Estimators
本文利用Edgeworth展开证明了小带宽渐近近似导出的密度加权平均导数推断程序具有更优的高阶分布性质，优于基于渐近线性近似的程序，为模拟中的有限样本性能提供了理论依据。
In 1991 Stute introduced a class of estimators called conditional U-statistics. They can be seen as a generalization of the Nadaraya-Watson estimator, and their strong pointwise consistency to the general regression function has been obtained in the same paper by Stute. Very recently, Gine and Mason introduced the notion of a local U-process, which generalizes that of a local empirical process, and obtained central limit theorems and laws of the iterated logarithm for this class. We apply the methods developed by Einmahl and Mason (2005) and Gine and Mason (2007a,b) to establish uniform in ban…
In this paper we propose a variable bandwidth kernel regression estimator for $i.i.d.$ observations in $\mathbb{R}^2$ to improve the classical Nadaraya-Watson estimator. The bias is improved to the order of $O(h_n^4)$ under the condition that the fifth order derivative of the density function and the sixth order derivative of the regression function are bounded and continuous. We also establish the central limit theorems for the proposed ideal and true variable kernel regression estimators. The simulation study confirms our results and demonstrates the advantage of the variable bandwidth kerne…
We consider a nonparametric version of the integer-valued GARCH(1,1) model for time series of counts. The link function in the recursion for the variances is not specified by finite-dimensional parameters, but we impose nonparametric smoothness conditions. We propose a least squares estimator for this function and show that it is consistent with a rate that we conjecture to be nearly optimal.
Robust Inference for Convex Pairwise Difference Estimators
针对凸成对差分估计量，本文发展了分布理论和基于自助法的推断方法，通过小带宽渐近理论、广义刀切去偏和调整带宽方差扭曲的自助法，实现了在更弱带宽条件下的有效推断。
The angular power spectrum of a stationary random field on the sphere is estimated from the needlet coefficients of a single realization, observed with increasingly fine resolution. The estimator we consider is similar to the one recently used in practice by (Faÿ et al. 2008) to estimate the power spectrum of the Cosmic Microwave Background. The consistency of the estimator, in the asymptotics of high frequencies, is proved for a model with a stationary Gaussian field corrupted by heteroscedastic noise and missing data.
Robust estimation under multivariate normal (MVN) mixture model is always a computational challenge. A recently proposed maximum pseudo \b{eta}-likelihood estimator aims to estimate the unknown parameters of a MVN mixture model in the spirit of minimum density power divergence (DPD) methodology but with a relatively simpler and tractable computational algorithm even for larger dimensions. In this letter, we will rigorously derive the existence and weak consistency of the maximum pseudo \b{eta}-likelihood estimator in case of MVN mixture models under a reasonable set of assumptions.
We obtain a limit of a hierarchical Bayes estimator of a finite population mean when the sample size is large. The limit is in the sense of ordinary calculus, where the sample observations are treated as fixed quantities. Our result suggests a simple way to correct the hierarchical Bayes estimator to achieve design-consistency, a well-known property in the traditional randomization approach to finite population sampling. We also suggest three different measures of uncertainty of our proposed estimator.
Generalized linear mixed models are powerful tools for analyzing clustered data, where the unknown parameters are classically (and most commonly) estimated by the maximum likelihood and restricted maximum likelihood procedures. However, since the likelihood based procedures are known to be highly sensitive to outliers, M-estimators have become popular as a means to obtain robust estimates under possible data contamination. In this paper, we prove that, for sufficiently smooth general loss functions defining the M-estimators in generalized linear mixed models, the tail probability of the deviat…
Information Criterion-Based Rank Estimation Methods for Factor Analysis: A Unified Selection Consistency Theorem and Numerical Comparison
本文研究了因子模型中基于信息准则的秩估计量的选择一致性，建立了统一的选择一致性定理，并通过数值模拟验证了间隙条件对秩选择一致性的决定性作用，同时比较了不同估计量的优劣。
Motivated by the latest effort to employ banded matrices to estimate a high-dimensional covariance $Σ$, we propose a test for $Σ$ being banded with possible diverging bandwidth. The test is adaptive to the "large $p$, small $n$" situations without assuming a specific parametric distribution for the data. We also formulate a consistent estimator for the bandwidth of a banded high-dimensional covariance matrix. The properties of the test and the bandwidth estimator are investigated by theoretical evaluations and simulation studies, as well as an empirical analysis on a protein mass spectroscopy …
Averaging provides an alternative to bandwidth selection for density kernel estimation. We propose a procedure to combine linearly several kernel estimators of a density obtained from different, possibly data-driven, bandwidths. The method relies on minimizing an easily tractable approximation of the integrated square error of the combination. It provides, at a small computational cost, a final solution that improves on the initial estimators in most cases. The average estimator is proved to be asymptotically as efficient as the best possible combination (the oracle), with an error term that d…
We consider a one dimensional ballistic random walk evolving in an i.i.d. parametric random environment. We provide a maximum likelihood estimation procedure of the environment parameters based on a single observation of the path till the time it reaches a distant site, and prove that this estimator is consistent as the distant site tends to infinity. We also explore the numerical performances of our estimation procedure.
Consistency of the maximum likelihood and variational estimators in a dynamic stochastic block model
该研究针对动态随机块模型，证明了其参数的极大似然估计与变分估计在节点数和时间步增长时的一致性，给出收敛速率上界，并分析了时间步固定的特殊情形。
Consider an $(L,1)$ random walk in an i.i.d. random environment, whose environment involves certain parameter. We get the maximum likelihood estimator(MLE) of the environment parameter which can be written as functionals of a multitype branching process with immigration in a random environment(BPIRE). Because the offspring distributions of the involved multitype BPIRE are of the linear fractional type, the limit invariant distribution of the multitype BPIRE can be computed explicitly. As a result, we get the consistency of the MLE. Our result is a generalization of Comets et al. [Stochastic Pr…
Dynamic systems are ubiquitous in nature and are used to model many processes in biology, chemistry, physics, medicine, and engineering. In particular, systems of ordinary differential equations are commonly used for the mathematical modelling of the rate of change of dynamic processes. In many practical applications, the process can only be partially measured, a fact that renders estimation of parameters of the system extremely challenging. Recently, a 'direct integral estimator' for partially observed systems of ordinary differential equations was introduced. The practical performance of the…
This paper highlights a tension between semiparametric efficiency and bootstrap consistency in the context of a canonical semiparametric estimation problem, namely the problem of estimating the average density. It is shown that although simple plug-in estimators suffer from bias problems preventing them from achieving semiparametric efficiency under minimal smoothness conditions, the nonparametric bootstrap automatically corrects for this bias and that, as a result, these seemingly inferior estimators achieve bootstrap consistency under minimal smoothness conditions. In contrast, several "debi…
Strong Consistency of the SIMEX Estimator in Linear Regression with a Conditionally Poisson Covariate
针对带条件泊松型测量误差协变量的线性回归估计问题，本文提出拓展至条件泊松场景的POI-SIMEX算法，无需内部验证数据即可稳健校正误差，并证明了该估计量的强相合性。
A hidden Markov model with trends is a hidden Markov model whose emission distributions are translated by a trend that depends on the current hidden state and on the current time. Contrary to standard hidden Markov models, such processes are not homogeneous and cannot be made homogeneous by a simple de-trending step. We show that when the trends are polynomial, the maximum likelihood estimator is able to recover the trends together with the other parameters and is strongly consistent. More precisely, the supremum norm of the difference between the true trends and the estimated ones tends to ze…
Consistency of M-estimators for non-identically distributed data: the case of fixed-design distributional regression
本文拓展了非独立同分布数据下M估计量的相合性研究，针对三角阵列数据（含固定协变量分布回归）建立基本条件，解决了极值统计中参数依赖支撑集导致准则函数取$-\infty$的问题，支持恰当评分规则和条件极大似然等估计方法。
In this paper we introduce an estimator for the number of communities in the Stochastic Block Model (SBM), based on the maximization of a penalized version of the so-called Krichevsky-Trofimov mixture distribution. We prove its eventual almost sure convergence to the underlying number of communities, without assuming a known upper bound on that quantity. Our results apply to both the dense and the sparse regimes. To our knowledge this is the first consistency result for the estimation of the number of communities in the SBM in the unbounded case, that is when the number of communities is allow…
We define a new bandwidth-dependent kernel density estimator that improves existing convergence rates for the bias, and preserves that of the variation, when the error is measured in $L_1$. No additional assumptions are imposed to the extant literature.
Estimating the effective dimension reduction (EDR) space, related to the semiparametric regression model introduced by Li \cite{sir}, is based on the estimation of the covariance matrix $Λ$ of the conditional expectation of the vector of predictors given the response. An estimator $\widehatΛ_n$ of $Λ$ based on kernel method was introduced by Zhu and Fang \cite{Asymptotics} who then derived, under some conditions, the asymptotic distribution of $\sqrt{n}\left(\widehatΛ_n-Λ\right)$, as $n\rightarrow +\infty$. In this paper, we obtain, under specified conditions, the almost sure convergence of $\…
We present a greedy method for simultaneously performing local bandwidth selection and variable selection in nonparametric regression. The method starts with a local linear estimator with large bandwidths, and incrementally decreases the bandwidth of variables for which the gradient of the estimator with respect to bandwidth is large. The method--called rodeo (regularization of derivative expectation operator)--conducts a sequence of hypothesis tests to threshold derivatives, and is easy to implement. Under certain assumptions on the regression function and sampling density, it is shown that t…
We propose an adaptive estimator for the stationary distribution of a bifurcating Markov Chain on $\mathbb R^d$. Bifurcating Markov chains (BMC for short) are a class of stochastic processes indexed by regular binary trees. A kernel estimator is proposed whose bandwidth is selected by a method inspired by the works of Goldenshluger and Lepski [18]. Drawing inspiration from dimension jump methods for model selection, we also provide an algorithm to select the best constant in the penalty.
In this paper we consider two closely related problems : estimation of eigenvalues and eigenfunctions of the covariance kernel of functional data based on (possibly) irregular measurements, and the problem of estimating the eigenvalues and eigenvectors of the covariance matrix for high-dimensional Gaussian vectors. In Peng and Paul (2007), a restricted maximum likelihood (REML) approach has been developed to deal with the first problem. In this paper, we establish consistency and derive rate of convergence of the REML estimator for the functional data case, under appropriate smoothness conditi…
扫码添加微信好友，提出您的宝贵建议 👇
💡 备注请填写：网站反馈
发表机构由 AI 从论文文本抽取，如发现错误，欢迎反馈修正建议。