预测、泛化和递归的基本限制：熵创新的观点
Fundamental Limits of Prediction, Generalization, and Recursion: An Entropic-Innovations Perspective
摘要：
在本文中，我们将研究预测的基本性能限制，有或没有边信息。更具体地说，我们推导出通用的下限上的$\mathcal{L}_p$规范的预测误差是有效的任何预测算法和任何数据分布。同时，我们结合信息论的熵分析和预测/估计理论的创新方法来描述条件（例如，有向信息或互信息）以实现边界。我们还调查的影响，在拟合（学习）问题的预测与侧信息的角度分析的基本限制的结果，以及递归算法的基本限制，通过查看它们作为广义预测问题。
英文摘要：
In this paper, we examine the fundamental performance limits of prediction, with or without side information. More specifically, we derive generic lower bounds on the $\mathcal{L}_p$ norms of the prediction errors that are valid for any prediction algorithms and for any data distributions. Meanwhile, we combine the entropic analysis from information theory and the innovations approach from prediction/estimation theory to characterize the conditions (in terms of, e.g., directed information or mutual information) to achieve the bounds. We also investigate the implications of the results in analyzing the fundamental limits of generalization in fitting (learning) problems from the perspective of prediction with side information, as well as the fundamental limits of recursive algorithms by viewing them as generalized prediction problems.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
