arXivDaily arXiv每日学术速递 周一至周五更新
arXiv physics/0007070physics.data-ancond-mat.dis-nncond-mat.othercs.LGnlin.AOq-bio.OT
收藏

可预测性、复杂性和学习

Predictability, complexity and learning

William Bialek, Ilya Nemenman, Naftali Tishby

摘要:

我们定义{ \em预测信息} $I_{\rm pred} (T)$为时间序列的过去和未来之间的互信息。在大观测时间的限制下,发现了三种性质不同的行为:$T$可以保持有限,几何增长,或作为分数幂律增长。如果时间序列允许我们学习具有有限数量参数的模型,那么$I_{\rm pred} (T)$将以计算模型空间维数的系数进行数学增长。与此相反,幂律增长与学习无限参数(或非参数)模型(如具有光滑约束的连续函数)有关。预测信息和复杂性度量之间存在联系,复杂性度量在学习理论和通过统计力学和动力系统理论对物理系统的分析中都有定义。此外,以同样的方式,熵提供了独特的措施,可用的信息与一些简单和合理的条件相一致,我们认为,发散部分的$I_{\rm pred} (T)$提供了独特的措施,动态的复杂性的时间序列。最后,我们讨论了这些想法如何在物理学,统计学和生物学中的不同问题中发挥作用。

英文摘要:

We define {\em predictive information} $I_{\rm pred} (T)$ as the mutual information between the past and the future of a time series. Three qualitatively different behaviors are found in the limit of large observation times $T$: $I_{\rm pred} (T)$ can remain finite, grow logarithmically, or grow as a fractional power law. If the time series allows us to learn a model with a finite number of parameters, then $I_{\rm pred} (T)$ grows logarithmically with a coefficient that counts the dimensionality of the model space. In contrast, power--law growth is associated, for example, with the learning of infinite parameter (or nonparametric) models such as continuous functions with smoothness constraints. There are connections between the predictive information and measures of complexity that have been defined both in learning theory and in the analysis of physical systems through statistical mechanics and dynamical systems theory. Further, in the same way that entropy provides the unique measure of available information consistent with some simple and plausible conditions, we argue that the divergent part of $I_{\rm pred} (T)$ provides the unique measure for the complexity of dynamics underlying a time series. Finally, we discuss how these ideas may be useful in different problems in physics, statistics, and biology.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑