计算符号源的熵率&一个与分布无关的极限定理
Computing Entropy Rate Of Symbol Sources & A Distribution-free Limit Theorem
摘要:
序列数据流的熵率自然地量化了生成过程的复杂性。因此,熵率波动可以用作识别信号源中的动态扰动的工具,并且可以在没有明确的背景噪声表征的情况下进行。然而,估计熵速率的现有技术算法具有明显缓慢的收敛;使得这种熵方法在实践中不可行。我们在这里提出了一个从根本上新的方法来估计熵率,这是证明收敛速度显着更快的输入数据长度方面,并被证明是有效的,在不同的应用范围从英语文本的熵率估计的混沌动力系统的复杂性估计。此外,熵估计的收敛速度不遵循任何标准的极限定理,并且报告的算法无法提供计算值的任何置信区间。利用概率自动机理论的连接,我们建立了一个收敛速度的$O(\log \vert s \vert/\sqrt[3]{\vert s \vert})$作为输入长度的函数$\vert s \vert$,然后产生明确的不确定性估计,以及所需的数据长度,以满足预先指定的置信区间。
英文摘要:
Entropy rate of sequential data-streams naturally quantifies the complexity of the generative process. Thus entropy rate fluctuations could be used as a tool to recognize dynamical perturbations in signal sources, and could potentially be carried out without explicit background noise characterization. However, state of the art algorithms to estimate the entropy rate have markedly slow convergence; making such entropic approaches non-viable in practice. We present here a fundamentally new approach to estimate entropy rates, which is demonstrated to converge significantly faster in terms of input data lengths, and is shown to be effective in diverse applications ranging from the estimation of the entropy rate of English texts to the estimation of complexity of chaotic dynamical systems. Additionally, the convergence rate of entropy estimates do not follow from any standard limit theorem, and reported algorithms fail to provide any confidence bounds on the computed values. Exploiting a connection to the theory of probabilistic automata, we establish a convergence rate of $O(\log \vert s \vert/\sqrt[3]{\vert s \vert})$ as a function of the input length $\vert s \vert$, which then yields explicit uncertainty estimates, as well as required data lengths to satisfy pre-specified confidence bounds.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。