Dirichlet先验平滑能解决Shannon熵估计问题吗？
Does Dirichlet Prior Smoothing Solve the Shannon Entropy Estimation Problem?
摘要：
Dirichlet先验广泛应用于离散分布和离散分布泛函的估计。在Shannon熵估计方面，一种方法是将Dirichlet先验平滑分布插入熵泛函中，而另一种方法是在平方误差的Dirichlet先验下计算熵的Bayes估计，这是条件期望。我们表明，在一般情况下，他们做\emph{not}提高了最大似然估计，插件的经验分布到熵功能。无论我们如何调整Dirichlet先验中的参数，这种方法都不能实现熵估计中的极小极大率，正如最近由Jiao，Venkat，Han和Weissman以及Wu和Yang所描述的那样。使用$n$样本的极小极大率最优估计器的性能基本上是\emph{at least}与使用$n\ln n$样本的Dirichlet平滑熵估计器一样好。我们利用正线性算子的近似理论来分析任意统计模型下一般泛函的插入式估计的偏差，从而进一步巩固了这两个领域之间的相互作用，这是由Jiao，Venkat，Han和Weissman彻底开发和利用的。我们建立了逼近理论中的新结果，并应用它们来分析Dirichlet先验平滑插入熵估计的偏差。偏差分析和近似理论之间的这种相互作用的相关性和后果远远超出了本文的具体问题设置。
英文摘要：
The Dirichlet prior is widely used in estimating discrete distributions and functionals of discrete distributions. In terms of Shannon entropy estimation, one approach is to plug-in the Dirichlet prior smoothed distribution into the entropy functional, while the other one is to calculate the Bayes estimator for entropy under the Dirichlet prior for squared error, which is the conditional expectation. We show that in general they do \emph{not} improve over the maximum likelihood estimator, which plugs-in the empirical distribution into the entropy functional. No matter how we tune the parameters in the Dirichlet prior, this approach cannot achieve the minimax rates in entropy estimation, as recently characterized by Jiao, Venkat, Han, and Weissman, and Wu and Yang. The performance of the minimax rate-optimal estimator with $n$ samples is essentially \emph{at least} as good as that of the Dirichlet smoothed entropy estimators with $n\ln n$ samples. We harness the theory of approximation using positive linear operators for analyzing the bias of plug-in estimators for general functionals under arbitrary statistical models, thereby further consolidating the interplay between these two fields, which was thoroughly developed and exploited by Jiao, Venkat, Han, and Weissman. We establish new results in approximation theory, and apply them to analyze the bias of the Dirichlet prior smoothed plug-in entropy estimator. This interplay between bias analysis and approximation theory is of relevance and consequence far beyond the specific problem setting in this paper.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
