需要多少样本才能利用平滑性?
How many samples are needed to leverage smoothness?
AI总结:
本文通过推导泛化误差的新下界,形式化了平滑性打破维数灾难所需的样本数量直觉,并研究了常数和过渡态在实践中的主导作用。
摘要:
统计学习的一个核心原则是,目标函数的平滑性可以打破维数灾难。然而,学习一个平滑函数似乎需要足够多的彼此接近的样本,才能获得高阶导数的有意义的估计,这在机器学习问题中可能很难实现,因为在这些问题中,数据数量与输入维数之比相对较小。通过推导泛化误差的新下界,本文形式化了这一直觉,然后研究了常数和过渡态的作用,这些在经典学习理论陈述中通常不被描述,但在实践中却起着主导作用。
英文摘要:
A core principle in statistical learning is that smoothness of target functions allows to break the curse of dimensionality. However, learning a smooth function seems to require enough samples close to one another to get meaningful estimate of high-order derivatives, which would be hard in machine learning problems where the ratio between number of data and input dimension is relatively small. By deriving new lower bounds on the generalization error, this paper formalizes such an intuition, before investigating the role of constants and transitory regimes which are usually not depicted beyond classical learning theory statements while they play a dominant role in practice.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。