Salim Bouzebda, Issam Elhattab
We establish uniform-in-bandwidth consistency for kernel-type estimators of the differential entropy. We consider two kernel-type estimators of Shannon's entropy. As a consequence, an asymptotic 100% confidence interval of entropy is provided.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
继续探索
Shannon's entropy is one of the building blocks of information theory and an essential aspect of Machine Learning methods (e.g., Random Forests). Yet, it is only finitely defined for distributions with fast decaying tails on a countable alphabet. The unboundedness of Shannon's entropy over the general class of all distributions on an alphabet prevents its potential utility from being fully realized. To fill the void in the foundation of information theory, Zhang (2020) proposed generalized Shannon's entropy, which is finitely defined everywhere. The plug-in estimator, adopted in almost all ent…
In this paper we study the ideal variable bandwidth kernel density estimator introduced by McKay (1993) and Jones, McKay and Hu (1994) and the plug-in practical version of the variable bandwidth kernel estimator with two sequences of bandwidths as in Giné and Sang (2013). Based on the bias and variance analysis of the ideal and true variable bandwidth kernel density estimators, we study the central limit theorems for each of them.
Nonparametric density and regression estimators commonly depend on a bandwidth. The asymptotic properties of these estimators have been widely studied when bandwidths are nonstochastic. In practice, however, in order to improve finite sample performance of these estimators, bandwidths are selected by data driven methods, such as cross-validation or plug-in procedures. As a result nonparametric estimators are usually constructed using stochastic bandwidths. In this paper we establish the asymptotic equivalence in probability of local polynomial regression estimators under stochastic and nonstoc…
A New Estimator of Kullback--Leibler Divergence via Shannon Entropy
本文提出了一种基于香农熵的KL散度估计器，通过k近邻方法改进了KL散度估计，并通过蒙特卡罗实验验证了其在高维数据中优于传统正态性检验的性能。
In a regression model, we write the Nadaraya-Watson estimator of the regression function as the quotient of two kernel estimators, and propose a bandwidth selection method for both the numerator and the denominator. We prove risk bounds for both data driven estimators and for the resulting ratio. The simulation study confirms that both estimators have good performances, compared to the ones obtained by cross-validation selection of the bandwidth. However, unexpectedly, the single-bandwidth cross-validation estimator is found to be much better than the ratio of the previous two good estimators,…
In this paper we establish the uniform in bandwidth consistency for the transformation kernel estimator of copulas introduced in [Omelka et al.(2009)]. To this end, we first prove a uniform in bandwidth law of the iterated logarithm for the maximal deviation of this estimator from its expectation. We then show that, as n goes to infinity, the bias of the estimator converges to zero uniformly in the bandwidth h, varying over a suitable interval. A practical method of selecting the optimal bandwidth is also presented. Finally, we make conclusive simulation experiments showing the performance of …
In this paper we propose an automatic selection of the bandwidth of the semi-recursive kernel estimators of a regression function defined by the stochastic approximation algorithm. We showed that, using the selected bandwidth and some special stepsizes, the proposed semi-recursive estimators will be very competitive to the nonrecursive one in terms of estimation error but much better in terms of computational costs. We corroborated these theoretical results through simulation study and a real dataset.
In this paper we consider the kernel estimators of a distribution function defined by the stochastic approximation algorithm when the observation are contamined by measurement errors. It is well known that this estimators depends heavily on the choice of a smoothing parameter called the bandwidth. We propose a specific second generation plug-in method of the deconvolution kernel distribution estimators defined by the stochastic approximation algorithm. We show that, using the proposed bandwidth selection and the stepsize which minimize the MISE (Mean Integrated Squared Error), the proposed est…
A note on kernel density estimators with optimal bandwidths
本文研究了最优带宽核密度估计器的累积分布函数，证明其在样本量增加时几乎必然位于经验分布函数的置信区间外。
A method of estimating the joint probability mass function of a pair of discrete random variables is described. This estimator is used to construct the conditional Shannon-Réyni-Tsallis entropies estimates. From there almost sure rates of convergence and asymptotic normality are established. The theorical results are validated by simulations.
We investigate the asymptotic mean squared error of kernel estimators of the intensity function of a spatial point process. We show that when $n$ independent copies of a point process in $\mathbb R^d$ are superposed, the optimal bandwidth $h_n$ is of the order $n^{-1/(d+4)}$ under appropriate smoothness conditions on the kernel and true intensity function. We apply the Abramson principle to define adaptive kernel estimators and show that asymptotically the optimal adaptive bandwidth is of the order $n^{-1/(d+8)}$ under appropriate smoothness conditions.
Asymptotic Normality and Convergence Rates for Tsallis Entropy Estimators via Stabilization Techniques
该研究针对一般度量测度空间上的Tsallis熵估计量，采用稳定化技术建立其渐近正态性与收敛速率，扩展了Shannon和Rényi熵的相关结果，为复杂空间和高维场景的非参数统计推断提供了有效框架。
An effective estimation of multivariate density functions using extended-beta kernels with Bayesian adaptive bandwidths
针对多元密度函数估计问题，提出带贝叶斯自适应带宽的多重扩展β核（MEBK）估计器，建立其渐近性质并提供带宽选择方法，经模拟与真实数据验证，在灵活性和通用性上优于高斯、伽马核估计器。
Given an i.i.d sample $(Y_i,Z_i)$, taking values in $\RRR^{d'}\times \RRR^d$, we consider a collection Nadarya-Watson kernel estimators of the conditional expectations $\EEE(<c_g(z),g(Y)>+d_g(z)\mid Z=z)$, where $z$ belongs to a compact set $H\subset \RRR^d$, $g$ a Borel function on $\RRR^{d'}$ and $c_g(\cdot),d_g(\cdot)$ are continuous functions on $\RRR^d$. Given two bandwidth sequences $h_n<\wth_n$ fulfilling mild conditions, we obtain an exact and explicit almost sure limit bounds for the deviations of these estimators around their expectations, uniformly in $g\in\GG,\;z\in H$ and $h_n\le …
In this paper we propose an automatic bandwidth selection of the recursive kernel density estimators with missing data in the context of global and local density estimation. We showed that, using the selected bandwidth and a special stepsize, the proposed recursive estimators outperformed the nonrecursive one in terms of estimation error in the case of global estimation. However, the recursive estimators are much better in terms of computational costs. We corroborated these theoretical results through simulation studies and on the simulated data of the Aquitaine cohort of HIV-1 infected patien…
Estimation of Shannon differential entropy: An extensive comparative review
本研究通过蒙特卡洛模拟比较了45种基于间距、KDE和kNN的Shannon微分熵估计器，发现间距法在单变量下最优但多变量下不存在，kNN虽劣但适用全维度，并推荐了不同维度的最优估计器。
In this paper we propose a variable bandwidth kernel regression estimator for $i.i.d.$ observations in $\mathbb{R}^2$ to improve the classical Nadaraya-Watson estimator. The bias is improved to the order of $O(h_n^4)$ under the condition that the fifth order derivative of the density function and the sixth order derivative of the regression function are bounded and continuous. We also establish the central limit theorems for the proposed ideal and true variable kernel regression estimators. The simulation study confirms our results and demonstrates the advantage of the variable bandwidth kerne…
Averaging provides an alternative to bandwidth selection for density kernel estimation. We propose a procedure to combine linearly several kernel estimators of a density obtained from different, possibly data-driven, bandwidths. The method relies on minimizing an easily tractable approximation of the integrated square error of the combination. It provides, at a small computational cost, a final solution that improves on the initial estimators in most cases. The average estimator is proved to be asymptotically as efficient as the best possible combination (the oracle), with an error term that d…
Robust Inference for Convex Pairwise Difference Estimators
针对凸成对差分估计量，本文发展了分布理论和基于自助法的推断方法，通过小带宽渐近理论、广义刀切去偏和调整带宽方差扭曲的自助法，实现了在更弱带宽条件下的有效推断。
In 1991 Stute introduced a class of estimators called conditional U-statistics. They can be seen as a generalization of the Nadaraya-Watson estimator, and their strong pointwise consistency to the general regression function has been obtained in the same paper by Stute. Very recently, Gine and Mason introduced the notion of a local U-process, which generalizes that of a local empirical process, and obtained central limit theorems and laws of the iterated logarithm for this class. We apply the methods developed by Einmahl and Mason (2005) and Gine and Mason (2007a,b) to establish uniform in ban…
We define a new bandwidth-dependent kernel density estimator that improves existing convergence rates for the bias, and preserves that of the variation, when the error is measured in $L_1$. No additional assumptions are imposed to the extant literature.
We present a greedy method for simultaneously performing local bandwidth selection and variable selection in nonparametric regression. The method starts with a local linear estimator with large bandwidths, and incrementally decreases the bandwidth of variables for which the gradient of the estimator with respect to bandwidth is large. The method--called rodeo (regularization of derivative expectation operator)--conducts a sequence of hypothesis tests to threshold derivatives, and is easy to implement. Under certain assumptions on the regression function and sampling density, it is shown that t…
Multivariate associated kernel estimators, which depend on both target point and bandwidth matrix, are appropriate for partially or totally bounded distributions and generalize the classical ones as Gaussian. Previous studies on multivariate associated kernels have been restricted to product of univariate associated kernels, also considered having diagonal bandwidth matrices. However, it is shown in classical cases that for certain forms of target density such as multimodal, the use of full bandwidth matrices offers the potential for significantly improved density estimation. In this paper, ge…
We propose an adaptive estimator for the stationary distribution of a bifurcating Markov Chain on $\mathbb R^d$. Bifurcating Markov chains (BMC for short) are a class of stochastic processes indexed by regular binary trees. A kernel estimator is proposed whose bandwidth is selected by a method inspired by the works of Goldenshluger and Lepski [18]. Drawing inspiration from dimension jump methods for model selection, we also provide an algorithm to select the best constant in the penalty.
We address the problem of density estimation with $\mathbb{L}_s$-loss by selection of kernel estimators. We develop a selection procedure and derive corresponding $\mathbb{L}_s$-risk oracle inequalities. It is shown that the proposed selection rule leads to the estimator being minimax adaptive over a scale of the anisotropic Nikol'skii classes. The main technical tools used in our derivations are uniform bounds on the $\mathbb{L}_s$-norms of empirical processes developed recently by Goldenshluger and Lepski [Ann. Probab. (2011), to appear].
We propose a new fully non-parametric two-step adaptive bandwidth selection method for kernel estimators of spatial point process intensity functions based on the Campbell-Mecke formula and Abramson's square root law. We present a simulation study to assess its performance relative to the Cronie-Van Lieshout global bandwidth selector and apply the technique to data on induced earthquakes in theGroningen gas field.
We discuss and compare various approaches to the problem of bandwidth selection for kernel estimators of intensity functions of spatial point processes. We also propose a new method based on the Campbell formula applied to the reciprocal intensity function. The new method is fully non-parametric, does not require knowledge of the product densities, and is not restricted to a specific class of point process models.
Suppose that $n$ statistical units are observed, each following the model $Y(x_j)=m(x_j)+ ε(x_j),\, j=1,...,N,$ where $m$ is a regression function, $0 \leq x_1 <...<x_N \leq 1$ are observation times spaced according to a sampling density $f$, and $ε$ is a continuous-time error process having mean zero and regular covariance function. Considering the local polynomial estimation of $m$ and its derivatives, we derive asymptotic expressions for the bias and variance as $n,N\to\infty$. Such results are particularly relevant in the context of functional data where essential information is contained …
We present a greedy method for simultaneously performing local bandwidth selection and variable selection in nonparametric regression. The method starts with a local linear estimator with large bandwidths, and incrementally decreases the bandwidth of variables for which the gradient of the estimator with respect to bandwidth is large. The method--called rodeo (regularization of derivative expectation operator)--conducts a sequence of hypothesis tests to threshold derivatives, and is easy to implement. Under certain assumptions on the regression function and sampling density, it is shown that t…
We derive asymptotic normality of kernel type deconvolution estimators of the density, the distribution function at a fixed point, and of the probability of an interval. We consider the so called super smooth case where the characteristic function of the known distribution decreases exponentially. It turns out that the limit behavior of the pointwise estimators of the density and distribution function is relatively straightforward while the asymptotics of the estimator of the probability of an interval depends in a complicated way on the sequence of bandwidths.
The celebrated Nadaraya-Watson kernel estimator is among the most studied method for nonparametric regression. A classical result is that its rate of convergence depends on the number of covariates and deteriorates quickly as the dimension grows, which underscores the "curse of dimensionality" and has limited its use in high dimensional settings. In this article, we show that when the true regression function is single or multi-index, the effects of the curse of dimensionality may be mitigated for the Nadaraya-Watson kernel estimator. Specifically, we prove that with $K$-fold cross-validation,…
Warped Kernel Estimator for I.I.D. Paths of Diffusion Processes
针对扩散过程漂移函数估计问题，本文提出基于独立连续观测的非参数扭曲核估计量，证明其风险界，扩展 PCO 带宽选择方法，并通过数值实验验证。
Nonparametric incidence estimation and bootstrap bandwidth selection in mixture cure models
本研究提出混合治愈模型的完全非参数估计方法，推导发生率估计量的性质与渐近最优带宽，给出自助法带宽选择方案，模拟验证其性能并应用于结直肠癌临床数据。
The present paper studies density deconvolution in the presence of small Berkson errors, in particular, when the variances of the errors tend to zero as the sample size grows. It is known that when the Berkson errors are present, in some cases, the unknown density estimator can be obtain by simple averaging without using kernels. However, this may not be the case when Berkson errors are asymptotically small. By treating the former case as a kernel estimator with the zero bandwidth, we obtain the optimal expressions for the bandwidth. We show that the density of Berkson errors acts as a regular…
We establish sufficient conditions for the asymptotic normality of kernel density estimators, applied to causal linear random fields. Our conditions on the coefficients of linear random fields are weaker than known results, although our assumption on the bandwidth is not minimal. The proof is based on the $m$-approximation method. As a key step, we prove a central limit theorem for triangular arrays of stationary $m$-dependent random fields with unbounded $m$. We also apply a moment inequality recently established for stationary random fields.
Adaptation to inhomogeneous smoothness for densities with irregularities
针对带孤立不规则性的密度估计问题，构建非恒定带宽核估计器以获得更快的$L_p$收敛速度并证明其最优性，还提出受Lepski方法启发的自适应可变带宽选择方法，无需先验光滑性信息。
A scheme for locally adaptive bandwidth selection is proposed which sensitively shrinks the bandwidth of a kernel estimator at lowest density regions such as the support boundary which are unknown to the statistician. In case of a Hölder continuous density, this locally minimax-optimal bandwidth is shown to be smaller than the usual rate, even in case of homogeneous smoothness. Some new type of risk bound with respect to a density-dependent standardized loss of this estimator is established. This bound is fully nonasymptotic and allows to deduce convergence rates at lowest density regions that…
Weak-Curvature AMISE and Plug-in Bandwidth Selection for Kernel Density Estimation
本文研究了在$C^{1,1}$正则性条件下核密度估计的AMISE理论，提出了弱曲率概念，并在不假设经典二阶导数连续的情况下，推导出AMISE公式、最优带宽和Epanechnikov核最优性。
In this paper, we propose to construct confidence bands by bootstrapping the debiased kernel density estimator (for density estimation) and the debiased local polynomial regression estimator (for regression analysis). The idea of using a debiased estimator was recently employed by Calonico et al. (2018b) to construct a confidence interval of the density function (and regression function) at a given point by explicitly estimating stochastic variations. We extend their ideas of using the debiased estimator and further propose a bootstrap approach for constructing simultaneous confidence bands. T…
Data driven extreme value distribution estimation: Derivation of the Mean Integrated Squared Error, optimal bandwidth selection and stability conditions
本文提出了一种数据驱动的极值分布估计器，推导了其均方误差，用于计算最优带宽并建立了带宽优化过程的稳定性条件。
An exact, closed form, and easy to compute expression for the mean integrated squared error (MISE) of a kernel estimator of a normal mixture cumulative distribution function is derived for the class of arbitrary order Gaussian-based kernels. Comparisons are made with MISE of the empirical distribution function, the infeasible minimum MISE of kernel estimators, and the asymptotically optimal second order uniform kernel. The results afford straightforward extensions to other classes of kernel functions and distributions. The analysis also offers a guide on when to use higher order kernels in dis…
The problem of fast computation of multivariate kernel density estimation (KDE) is still an open research problem. In our view, the existing solutions do not resolve this matter in a satisfactory way. One of the most elegant and efficient approach utilizes the fast Fourier transform. Unfortunately, the existing FFT-based solution suffers from a serious limitation, as it can accurately operate only with the constrained (i.e., diagonal) multivariate bandwidth matrices. In this paper we describe the problem and give a satisfactory solution. The proposed solution may be successfully used also in o…
Estimation of Integrated Volatility Functionals with Kernel Spot Volatility Estimators
本文针对多维伊藤半鞅的积分波动率泛函估计问题，提出采用通用核瞬时波动率估计量的插件式估计方法，建立相关中心极限定理，证明其能显著降低偏差且对带宽选择具有稳健性。
We study the kernel estimator of the transition density of bifurcating Markov chains. Under some ergodic and regularity properties, we prove that this estimator is consistent and asymptotically normal. Next, in the numerical studies, we propose two data-driven methods to choose the bandwidth parameters. These methods are based on the so-called two bandwidths approach.
When collections of functional data are too large to be exhaustively observed, survey sampling techniques provide an effective way to estimate global quantities such as the population mean function. Assuming functional data are collected from a finite population according to a probabilistic sampling scheme, with the measurements being discrete in time and noisy, we propose to first smooth the sampled trajectories with local polynomials and then estimate the mean function with a Horvitz-Thompson estimator. Under mild conditions on the population size, observation times, regularity of the trajec…
In the this paper, the authors propose to estimate the density of a targeted population with a weighted kernel density estimator (wKDE) based on a weighted sample. Bandwidth selection for wKDE is discussed. Three mean integrated squared error based bandwidth estimators are introduced and their performance is illustrated via Monte Carlo simulation. The least-squares cross-validation method and the adaptive weight kernel density estimator are also studied. The authors also consider the boundary problem for interval bounded data and apply the new method to a real data set subject to informative c…
The kedd package providing additional smoothing techniques to the R statistical system. Although various packages on the Comprehensive R Archive Network (CRAN) provide functions useful to nonparametric statistics, kedd aims to serve as a central location for more specifically of a nonparametric functions and data sets. The current feature set of the package can be split in four main categories: compute the convolutions and derivatives of a kernel function, compute the kernel estimators for a density of probability and its derivatives, computing the bandwidth selectors with different methods, d…
We prove a uniform in bandwidth law of the iterated logarithm for the maximal deviation of kernel copula estimators from their expectations. We deal especially with the \textit{local linear}, the \textit{mirror-reflection} and the \textit{transformation} estimators. These results are useful for establishing the strong uniform in bandwidth consistency of these kernel estimators.
Estimators of information theoretic measures such as entropy and mutual information are a basic workhorse for many downstream applications in modern data science. State of the art approaches have been either geometric (nearest neighbor (NN) based) or kernel based (with a globally chosen bandwidth). In this paper, we combine both these approaches to design new estimators of entropy and mutual information that outperform state of the art methods. Our estimator uses local bandwidth choices of $k$-NN distances with a finite $k$, independent of the sample size. Such a local and data dependent choic…
Length-of-stay times in hospital for COVID-19 patients using the smoothed Beran's estimator with bootstrap bandwidth selection
本文针对COVID-19患者病房和ICU住院时长，提出基于双重平滑Beran估计量和自助带宽选择的条件生存函数估计方法，并通过模拟验证其良好性能。
We consider estimating the density of a response conditioning on an error-prone covariate. Motivated by two existing kernel density estimators in the absence of covariate measurement error, we propose a method to correct the existing estimators for measurement error. Asymptotic properties of the resultant estimators under different types of measurement error distributions are derived. Moreover, we adjust bandwidths readily available from existing bandwidth selection methods developed for error-free data to obtain bandwidths for the new estimators. Extensive simulation studies are carried out t…
Finite-Sample Bias Correction for Plug-in Estimators of Extropy, Rényi Extropy, and Tsallis Extropy
本文针对有限支撑离散随机变量的三类外熵插件估计器推导一阶偏差公式，提出偏差校正方法，经模拟验证其有限样本性能更优，支持渐近场景中使用未校正插件估计器。
Maximum likelihood estimation of a log-concave probability density is formulated as a convex optimization problem and shown to have an equivalent dual formulation as a constrained maximum Shannon entropy problem. Closely related maximum Renyi entropy estimators that impose weaker concavity restrictions on the fitted density are also considered, notably a minimum Hellinger discrepancy estimator that constrains the reciprocal of the square-root of the density to be concave. A limiting form of these estimators constrains solutions to the class of quasi-concave densities.
We propose nonparametric estimation of divergence measures between continuous distributions. Our approach is based on a plug-in kernel- type estimators of density functions. We give the uniform in bandwidth consistency for the proposal estimators. As a consequence, their asymp- totic 100% confidence intervals are also provided.
It is well-known that kernel regression estimators do not produce a constant estimator variance over a domain. To correct this problem, Nishida and Kanazawa (2015) proposed a variance-stabilizing (VS) local variable bandwidth for Local Linear (LL) regression estimator. In contrast, Choi and Hall (1998) proposed the skewing (SK) methods for a univariate LL estimator and constructed a convex combination of one LL estimator and two SK estimators that are symmetrically placed on both sides of the LL estimator (the convex combination (CC) estimator) to eliminate higher-order terms in its asymptotic…
We consider the problem of estimating functionals of discrete distributions, and focus on tight nonasymptotic analysis of the worst case squared error risk of widely used estimators. We apply concentration inequalities to analyze the random fluctuation of these estimators around their expectations, and the theory of approximation using positive linear operators to analyze the deviation of their expectations from the true functional, namely their \emph{bias}. We characterize the worst case squared error risk incurred by the Maximum Likelihood Estimator (MLE) in estimating the Shannon entropy $H…
We propose completely nonparametric methodology to investigate location-scale modelling of two-component mixture cure models, where the responses of interest are only indirectly observable due to the presence of censoring and the presence of so-called long-term survivors that are always censored. We use covariate-localized nonparametric estimators, which depend on a bandwidth sequence, to propose an estimator of the error distribution function that has not been considered before in the literature. When this bandwidth belongs to a certain range of undersmoothing bandwidths, the asymptotic distr…
We first revisit the problem of estimating the spot volatility of an Itô semimartingale using a kernel estimator. We prove a Central Limit Theorem with optimal convergence rate for a general two-sided kernel. Next, we introduce a new pre-averaging/kernel estimator for spot volatility to handle the microstructure noise of ultra high-frequency observations. We prove a Central Limit Theorem for the estimation error with an optimal rate and study the optimal selection of the bandwidth and kernel functions. We show that the pre-averaging/kernel estimator's asymptotic variance is minimal for exponen…
A Fast, Closed-Form Bandwidth Selector for the Beta Kernel Density Estimator
本文提出一种快速闭合形式的Beta核密度估计器带宽选择器，通过方法-of-moments近似将带宽选择复杂度从迭代优化降低到O(1)，并在蒙特卡洛模拟和现实数据验证中表现出色。
We prove the asymptotic normality of the kernel density estimator (introduced by Rosenblatt (1956) and Parzen (1962)) in the context of stationary strongly mixing random fields. Our approach is based on the Lindeberg's method rather than on Bernstein's small-block-large-block technique and coupling arguments widely used in previous works on nonparametric estimation for spatial processes. Our method allows us to consider only minimal conditions on the bandwidth parameter and provides a simple criterion on the (non-uniform) strong mixing coefficients which do not depend on the bandwith.
扫码添加微信好友，提出您的宝贵建议 👇
💡 备注请填写：网站反馈
发表机构由 AI 从论文文本抽取，如发现错误，欢迎反馈修正建议。