从Hilbert空间中的非随机数据学习:最优恢复观点
Learning from Non-Random Data in Hilbert Spaces: An Optimal Recovery Perspective
摘要:
经典统计学习中的泛化概念通常与数据点是独立同分布(IID)随机变量的假设有关。虽然在许多应用程序中相关,但这一假设通常可能不成立,鼓励开发对非IID数据具有鲁棒性的学习框架。在这项工作中,我们考虑从最优恢复的角度回归问题。依赖于与选择假设类类似的模型假设,学习器的目标是最小化最坏情况下的错误,而不依赖于对数据的任何概率假设。首先,我们开发了一个半定的程序,用于计算有限维希尔伯特空间中任何恢复映射的最坏情况下的误差。然后,对于任何希尔伯特空间,我们表明,最优恢复提供了一个公式,这是用户友好的算法的观点,只要假设类是线性的。有趣的是,这个公式在某些情况下与核无岭回归相吻合,证明了最小化平均误差和最坏情况误差可以产生相同的解。我们提供了数值实验来支持我们的理论研究结果。
英文摘要:
The notion of generalization in classical Statistical Learning is often attached to the postulate that data points are independent and identically distributed (IID) random variables. While relevant in many applications, this postulate may not hold in general, encouraging the development of learning frameworks that are robust to non-IID data. In this work, we consider the regression problem from an Optimal Recovery perspective. Relying on a model assumption comparable to choosing a hypothesis class, a learner aims at minimizing the worst-case error, without recourse to any probabilistic assumption on the data. We first develop a semidefinite program for calculating the worst-case error of any recovery map in finite-dimensional Hilbert spaces. Then, for any Hilbert space, we show that Optimal Recovery provides a formula which is user-friendly from an algorithmic point-of-view, as long as the hypothesis class is linear. Interestingly, this formula coincides with kernel ridgeless regression in some cases, proving that minimizing the average error and worst-case error can yield the same solution. We provide numerical experiments in support of our theoretical findings.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。