arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2604.07233cs.LGcs.CC
收藏

机器学习如何管理复杂性?

How Does Machine Learning Manage Complexity?

Illinois Institute of Technology(伊利诺伊理工学院);Magdalen College, University of Oxford(牛津大学莫德林学院)

Lance Fortnow

AI总结:

本文从计算复杂性角度探讨机器学习模型处理复杂系统的能力,提出通过概率管理复杂性,并证明学习可计算分布能有效逼近均匀分布。

摘要:

我们提供一个计算复杂性视角来理解机器学习模型的威力,特别是其建模复杂系统的潜力。机器学习模型通常在可采样或更复杂的分布上训练,范围远超可计算分布。通过聚焦可计算分布,机器学习模型可通过概率更好地管理复杂性。我们抽象出特定学习机制,将机器学习建模为生成具有多项式界最大熵的P/poly可计算分布。我们通过展示,如果机器学习模型生成的分布μ在对抗加密伪随机生成器生成的分布时最小化误差,则μ必须接近均匀分布,来说明学习可计算分布如何建模复杂性。

英文摘要:

We provide a computational complexity lens to understand the power of machine learning models, particularly their ability to model complex systems. Machine learning models are often trained on data drawn from sampleable or more complex distributions, a far wider range of distributions than just computable ones. By focusing on computable distributions, machine learning models can better manage complexity via probability. We abstract away from specific learning mechanisms, modeling machine learning as producing P/poly-computable distributions with polynomially-bounded max-entropy. We illustrate how learning computable distributions models complexity by showing that if a machine learning model produces a distribution $μ$ that models the distribution generated by a cryptographic pseudorandom generator at least as well as the uniform distribution does, then $μ$ must be close to uniform. We view training as compression that removes the randomness from the data, keeping only its structure, and inference as adding randomness back to generate new outputs. A computationally bounded model cannot distinguish pseudorandomness from true randomness, so training removes both, treating anything it cannot efficiently predict as noise. The same holds at inference, where the randomness added back can be, and in practice is, pseudorandom. In the appendix, we make the connection to compression precise and, using a characterization of pseudoentropy by Vadhan and Zheng, show that the best loss achievable by an efficient model is the next-bit pseudoentropy of the data.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑