arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 1702.05222cs.ITcs.AImath.ITstat.ML
收藏

基于最近邻比的信息发散度直接估计

Direct Estimation of Information Divergence Using Nearest Neighbor Ratios

University of Michigan(密歇根大学);Yale University(耶鲁大学)

Morteza Noshad, Kevin R. Moon, Salimeh Yasaei Sekeh, Alfred O. Hero

摘要:

我们提出了一个直接估计方法的Rényi和f-散度措施的基础上一个新的图论解释。假设我们给出两个样本集$X$和$Y$,分别具有$N$和$M$样本,其中$η:=M/N$是常数值。考虑联合数据集$(X,Y)$中$Y$的$k$ -最近邻($k$ -NN)图,证明了所有$k$ -NN点中$X$点的个数与$Y$点的个数的平均幂比与$X$和$Y$密度的Rényi散度成正比.类似的方法也可以用于估计f-散度度量。我们推导出偏差率和方差率,并证明了对于$γ$ -Hölder光滑函数类,估计量达到了$O(N^{-2γ/(γ+d)})$的MSE率。此外,通过使用加权系综估计技术,密度函数的连续和有界导数的顺序$d$,和一些额外的条件,在支持集的边界,我们得到一个系综估计,实现参数MSE率$O(1/N)$。我们的估计量比其他竞争估计量更易于计算,这使得它们在许多实际应用中具有吸引力。

英文摘要:

We propose a direct estimation method for Rényi and f-divergence measures based on a new graph theoretical interpretation. Suppose that we are given two sample sets $X$ and $Y$, respectively with $N$ and $M$ samples, where $η:=M/N$ is a constant value. Considering the $k$-nearest neighbor ($k$-NN) graph of $Y$ in the joint data set $(X,Y)$, we show that the average powered ratio of the number of $X$ points to the number of $Y$ points among all $k$-NN points is proportional to Rényi divergence of $X$ and $Y$ densities. A similar method can also be used to estimate f-divergence measures. We derive bias and variance rates, and show that for the class of $γ$-Hölder smooth functions, the estimator achieves the MSE rate of $O(N^{-2γ/(γ+d)})$. Furthermore, by using a weighted ensemble estimation technique, for density functions with continuous and bounded derivatives of up to the order $d$, and some extra conditions at the support set boundary, we derive an ensemble estimator that achieves the parametric MSE rate of $O(1/N)$. Our estimators are more computationally tractable than other competing estimators, which makes them appealing in many practical applications.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑