通过有限空间上的测度集中性获得更紧的学习保证
Tighter Learning Guarantees on Digital Computers via Concentration of Measure on Finite Spaces
AI总结:
本文提出了一种适用于数字计算机的学习模型通用界限,通过调整参数m,实现了更紧的泛化界限,同时保持了维度无关的最坏情况速率。
摘要:
在欧几里得空间R^d中的机器学习模型在数字计算机上实现时,其泛化能力趋于收敛到0,其泛化间隙以c/N^{1/2}的速率收敛于样本量N。然而,通过经典方法获得的常数c>0在环境维度d和机器精度方面可能较大,当N较小时会带来挑战。本文推导了一组适用于数字计算机学习模型的泛化界限{c_m/N^{1/(2∨m)}}_{m=1}^∞,该界限适应于样本量N和所谓的几何表示维度m。根据N调整参数m可显著收紧实际样本量N的泛化界限,而设置m较小则保持了最优的维度无关最坏情况速率O(1/N^{1/2})。值得注意的是,对于离散欧几里得域上的学习模型,c_m∈O(m^{1/2})。此外,我们的自适应泛化界限是基于我们新非渐近结果的有限度量空间中的测度集中性结果,该结果通过利用度量嵌入论证建立。
英文摘要:
Machine learning models with inputs in a Euclidean space $\mathbb{R}^d$, when implemented on digital computers, generalize, and their generalization gap converges to $0$ at a rate of $c/N^{1/2}$ concerning the sample size $N$. However, the constant $c>0$ obtained through classical methods can be large in terms of the ambient dimension $d$ and machine precision, posing a challenge when $N$ is small to realistically large. In this paper, we derive a family of generalization bounds $\{c_m/N^{1/(2\vee m)}\}_{m=1}^{\infty}$ tailored for learning models on digital computers, which adapt to both the sample size $N$ and the so-called geometric representation dimension $m$ of the discrete learning problem. Adjusting the parameter $m$ according to $N$ results in significantly tighter generalization bounds for practical sample sizes $N$, while setting $m$ small maintains the optimal dimension-free worst-case rate of $\mathcal{O}(1/N^{1/2})$. Notably, $c_{m}\in \mathcal{O}(m^{1/2})$ for learning models on discretized Euclidean domains. Furthermore, our adaptive generalization bounds are formulated based on our new non-asymptotic result for concentration of measure in finite metric spaces, established via leveraging metric embedding arguments.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。