arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 1709.09699cs.ITmath.IT
收藏

隐马尔可夫模型Renyi熵的解析公式

Analytic Formulas for Renyi Entropy of Hidden Markov Models

Joachim Breitner, Maciej Skorski

摘要:

确定随机过程的熵率是一个基本的和困难的问题,封闭形式的解决方案,只有在特定的情况下。本文通过解决隐马尔可夫模型和Renyi熵的问题,推动了最新技术的发展。虽然马尔可夫链的问题简化为研究矩阵乘积的增长,但Hyndrome的计算涉及\emph{products of random matrices}。因此,这种情况要困难得多,到目前为止还没有明确的公式。我们展示了如何规避这个问题的整数阶的Renyi熵,减少问题再次到一个\emph{single matrix products}的矩阵形成的过渡和发射概率通过张量积。为了获得结果的渐近设置,我们使用一种新的技术来确定非负矩阵的权力的增长。经典的方法是Frobenius-Perron理论,但它需要正性假设;我们直接使用谱公式。因此,我们的结果不受限制,如不可约性和非周期性。这提高了我们对熵率的理解,即使是标准的(非隐藏的)马尔可夫链。最近发表的一个针对RSA的侧信道攻击被证明是有效的,使用我们的结果,专门为2阶。

英文摘要:

Determining entropy rates of stochastic processes is a fundamental and difficult problem, with closed-form solutions known only for specific cases. This paper pushes the state-of-the-art by solving the problem for Hidden Markov Models (HMMs) and Renyi entropies. While the problem for Markov chains reduces to studying the growth of a matrix product, computations for HMMs involve \emph{products of random matrices}. As a result, this case is much harder and no explicit formulas have been known so far. We show how to circumvent this issue for Renyi entropy of integer orders, reducing the problem again to a \emph{single matrix products} where the matrix is formed from transition and emission probabilities by means of tensor product. To obtain results in the asymptotic setting, we use a novel technique for determining the growth of non-negative matrix powers. The classical approach is the Frobenius-Perron theory, but it requires positivity assumptions; we instead work directly with the spectral formula. As a consequence, our results do not suffer from limitations such as irreducibility and aperiodicity. This improves our understanding of the entropy rate even for standard (unhidden) Markov chains. A recently published side-channel attack against RSA was proven effective using our result, specialized to order 2.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑