The statistical problem of using an initial sample to estimate the number of species in a larger sample has found important applications in fields far removed from ecology. Here we address the general problem of estimating the number of species that will be represented by at least a number r of observations in a future sample. The number r indicates species with sufficient observations, which are commonly used as a necessary condition for any robust statistical inference. We derive a procedure to construct consistent estimators that apply universally for a given population: once constructed, they can be evaluated as a simple function of r. Our approach is based on a relation between the number of species represented at least r times and the higher derivatives of the expected number of species discovered per unit of time. Combining this relation with a rational function approximation, we propose nonparametric estimators that are accurate for both large values of r and long-range extrapolations. We further show that our estimators retain asymptotic behaviors that are essential for applications on large-scale datasets. We evaluate the performance of this approach by both simulation and real data applications for inferences of the vocabulary of Shakespeare and Dickens, the topology of a Twitter social network, and molecular diversity in DNA sequencing data.
Finite Mixtures of Generalized Estimating Equations for Clustering Multivariate Correlated Outcomes
Faculty of Economics, Keio University(庆应义塾大学经济学部);Research School of Finance, Actuarial Studies and Statistics, The Australian National University(澳大利亚国立大学金融、精算与统计研究学院)
Local moment matching: A unified methodology for symmetric functional estimation and distribution estimation under Wasserstein distance
Stanford University(斯坦福大学)
我们提出\emph{Local Moment Matching (LMM)},对称函数估计和分布估计下瓦瑟斯坦距离的统一方法。我们构造了一个有效的可计算的估计,实现了极大极小率估计的分布排列,并表明,插件的方法,我们的无标签分布估计是“通用”的对称泛函估计的离散分布。而不是做最好的多项式逼近显式在现有文献中的功能估计,插件的方法进行多项式逼近隐式,并达到最佳的样本复杂性的熵,功率和支持大小泛函。
An Optimal Reduction of TV-Denoising to Adaptive Online Learning
UC Santa Barbara(加州大学圣巴巴拉分校)
我们考虑从$n$噪声样本中估计一个函数的问题,该样本的离散全变差(TV)由$C_n$限定。我们揭示了与强自适应在线学习(Daniely et al,2015)的看似不同的问题的深层联系,并提供了一种$O(n \log n)$时间算法,该算法在平方误差损失下获得了$\tilde O (n^{1/3}C_n^{2/3})$的近极小极大最优速率。所得到的算法在线运行,并最佳地适应未知的平滑参数$C_n$。这导致了一个新的和更通用的替代小波为基础的方法(1)自适应估计电视有界函数;(2)在线预测电视有界的趋势,在时间序列。
Bocconi Institute for Data Science and Analytics, Bocconi University(博科尼大学数据科学与分析研究所);Department of Economics, Università degli studi di Bergamo(贝加莫大学经济系);Department of Economics, Management and Statistics, University of Milano-Bicocca(米兰比可卡大学经济、管理与统计系);Department of Statistical Sciences, Università Cattolica del Sacro Cuore(天主教大学统计科学系)