发表机构
King Abdullah University of Science and Technology（阿卜杜拉国王科技大学）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
本研究从比特复杂度视角构建统一逼近框架，对比经典方法与神经网络的逼近速率，发现神经网络的优势源于函数类复杂度差异，根本限制是比特复杂度灾难而非维度灾难。
AI 中文摘要
传统逼近理论以参数数量或自由度衡量收敛速率，但实际计算在有限精度下进行，参数需用有限比特数编码，因此逼近效率应基于计算比特复杂度评估，其与底层函数类的度量熵存在内在关联。本研究构建了基于二进制编码与度量熵的统一逼近框架，分析了多项式逼近、稀疏网格、有限元等经典方法，以及浅层与深度神经网络，对比了它们在具有可比度量熵的函数类上的逼近速率。研究发现，以比特为单位评估时，多数经典方法通常劣于度量熵决定的内在极限，而神经网络方法可能表现出不同行为；当以比特而非参数衡量复杂度时，没有方法能从根本上超越经典方法达到的逼近阶。研究结果还表明，神经网络的诸多看似优势，包括与维度无关的速率和超收敛现象，源于函数类复杂度的差异而非架构本身的优越性。从这个意义上说，传统的维度灾难可能具有误导性，根本限制是由度量熵支配的比特复杂度灾难。
英文摘要
Traditional approximation theory measures convergence rates in terms of the number of parameters or degrees of freedom. However, practical computation operates under finite precision: parameters must be encoded using a finite number of bits. Therefore, approximation efficiency should be evaluated in terms of computational bit complexity, which is intrinsically connected to the metric entropy of the underlying function class. In this work, we develop a unified approximation framework based on binary encoding and metric entropy. We analyze classical methods (including polynomial approximation, sparse grids, and finite elements) as well as shallow and deep neural networks, and compare their approximation rates for function classes with comparable metric entropy. We observe that, when evaluated in terms of bits, most classical methods are in general suboptimal relative to the intrinsic limits dictated by metric entropy, while neural network methods may exhibit different behaviors. We show that when complexity is measured in bits rather than parameters, no method fundamentally exceeds the approximation order achieved by classical approaches. Our results also indicate that many seeming advantages of neural networks, including dimension-independent rates and superconvergence phenomena, stem from differences in function class complexity rather than intrinsic architectural superiority. In this sense, the traditional curse of dimensionality can be misleading; the fundamental limitation is instead a curse of bit complexity, governed by metric entropy.
