arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 1907.02095cs.ITmath.ITmath.STstat.TH
收藏

通过互信息和MMSE理解相变

Understanding Phase Transitions via Mutual Information and MMSE

Galen Reeves, Henry Pfister

摘要:

理解和解决高维推理问题的能力对于现代数据科学至关重要。本文通过信息论的视角来研究高维推理问题,并将重点放在标准线性模型上,作为一个典型的例子,它既丰富到足以实用,又简单到足以进行严格的研究。特别是,该模型可以表现出相变,其中模型参数的任意小的变化可以引起估计质量的大的变化。对于这个模型,可以使用统计物理学中的复制方法来研究最优推理的性能,但是直到最近,人们还不知道得到的公式是否正确。在本章中,我们将介绍标准线性模型及其与信息论的联系。我们还描述了该模型的副本预测,并概述了作者最近的证明,它是准确的。

英文摘要:

The ability to understand and solve high-dimensional inference problems is essential for modern data science. This article examines high-dimensional inference problems through the lens of information theory and focuses on the standard linear model as a canonical example that is both rich enough to be practically useful and simple enough to be studied rigorously. In particular, this model can exhibit phase transitions where an arbitrarily small change in the model parameters can induce large changes in the quality of estimates. For this model, the performance of optimal inference can be studied using the replica method from statistical physics but, until recently, it was not known if the resulting formulas were actually correct. In this chapter, we present a tutorial description of the standard linear model and its connection to information theory. We also describe the replica prediction for this model and outline the authors' recent proof that it is exact.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑