电路和神经网络上图灵机的PAC学习增益
PAC-learning gains of Turing machines over circuits and neural networks
摘要：
当前深度学习方法的许多应用需要注意的是需要大规模数据。Kolmogorov复杂性结果建议的一个改进是将最小描述长度原则应用于计算通用模型。我们研究了这种方法原则上可以带来的样本效率的潜在增益。我们使用多项式时间图灵机来表示计算上的通用模型和布尔电路来表示作用于有限精度数字的人工神经网络（ANN）。我们的分析揭示了我们的问题和计算复杂性结果之间的直接联系。我们提供了下限和上限之间的MDL应用图灵机，而不是人工神经网络的样本效率的潜在收益。我们的边界取决于要学习的布尔函数的输入的位大小。此外，我们突出了电路复杂性和这些紧密的经典开放问题之间的密切关系。
英文摘要：
A caveat to many applications of the current Deep Learning approach is the need for large-scale data. One improvement suggested by Kolmogorov Complexity results is to apply the minimum description length principle with computationally universal models. We study the potential gains in sample efficiency that this approach can bring in principle. We use polynomial-time Turing machines to represent computationally universal models and Boolean circuits to represent Artificial Neural Networks (ANNs) acting on finite-precision digits. Our analysis unravels direct links between our question and Computational Complexity results. We provide lower and upper bounds on the potential gains in sample efficiency between the MDL applied with Turing machines instead of ANNs. Our bounds depend on the bit-size of the input of the Boolean function to be learned. Furthermore, we highlight close relationships between classical open problems in Circuit Complexity and the tightness of these.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
