发表机构
Department of Computer Science; School of Computer and Cyber Sciences; Augusta University（计算机科学系; 计算机与电子科学学院; 奥古斯塔大学）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
研究从样本估计Renyi熵和最小熵的样本复杂度，通过最大经验频率和二元分组等方法给出不同情况下的上下界，刻画了\(k\)和\(\alpha>1\)时最小熵估计的样本复杂度，证明在高阶 regime 样本复杂度为\(\Theta_\varepsilon(k\log k)\)。
AI 中文摘要
从样本中估计熵在信息论和属性测试中至关重要。香农熵衡量平均不确定性，使用\(\Theta(k / \log k)\)个样本可在\(k\)符号字母表上估计到常数加法精度。最小熵仅取决于最可能的符号，两者都是\(\alpha\)阶Renyi熵\(H_\alpha\)的特殊情况。本文刻画了\(k\)和整数\(\alpha>1\)时最小熵和Renyi熵估计的样本复杂度，下限对非整数\(\alpha\geq1.001\)也成立。证明了最小熵估计到常数加法精度的样本复杂度为\(\Theta(k\log k)\)，上限通过最大经验频率和二元分组集中性得到，匹配下限通过在均匀随机位置隐藏稍重符号得到。对于\(2\leq\alpha\leq c_0\log k\)的整数，给出匹配固定精度界\(\Theta_{c_0}(\alpha k^{1 - 1 / \alpha})\)。对于\(1.001\leq\alpha\leq c_0\log k\)的实数，证明了均匀下限\(\Omega_{c_0}(\alpha k^{1 - 1 / \alpha})\)。最后，当\(\alpha\)是\(\log k\)的足够大倍数时，最小熵均匀逼近\(H_\alpha\)，结合最小熵界在高阶 regime 给出\(\Theta_\varepsilon(k\log k)\)样本复杂度。
英文摘要
Estimating entropy from samples is fundamental in information theory and property testing. Shannon entropy measures average uncertainty and can be estimated to constant additive accuracy over a $k$-symbol alphabet using $Θ(k/\log k)$ samples. Min-entropy depends only on the most likely symbol. Both are special cases of order-$α$ R'{e}nyi entropy, $H_α$. We characterize the sample complexity of estimating min-entropy and R'{e}nyi entropy for $k$ and integer $α>1$; our lower bounds also hold for noninteger $α\ge1.001$. We prove that min-entropy estimation to constant additive accuracy has sample complexity $Θ(k\log k)$. The upper bound uses the largest empirical frequency and concentration via dyadic grouping. The matching lower bound hides a slightly heavier symbol at a uniformly random location. Thus, min-entropy requires $Θ(\log^2 k)$ more samples than Shannon entropy and corrects a previously stated $Θ(k/\log k)$ characterization. For every integer $2\leα\le c_0\log k$, we prove the matching fixed-accuracy bound $Θ_{c_0}(αk^{1-1/α})$. Previous results gave $Ω_α(k^{1-1/α})$ for fixed integer $α>1$ and $O_{c_0}(α^2k^{1-1/α})$ for all integer $α>1$. Our upper bound analyzes an unbiased falling-factorial estimator based on $α$-way collisions, while a hidden-heavy-coordinate construction gives the matching lower bound and shows that the factor $α$ is unavoidable. For every real $1.001\leα\le c_0\log k$, we prove the uniform lower bound $Ω_{c_0}(αk^{1-1/α})$. Finally, since $0\le H_α(p)-H_\infty(p)\le\log k/(α-1)$, min-entropy uniformly approximates $H_α$ when $α$ is a sufficiently large multiple of $\log k$. Combining this reduction with our min-entropy bounds gives $Θ_\varepsilon(k\log k)$ sample complexity in the high-order regime.
