进入危险区：高维函数与算子学习中的稳定外推

中文摘要 AI 辅助

分布外（OOD）泛化是科学机器学习中的一个核心挑战。我们研究测试分布与训练分布不同的回归问题，并追问：在目标函数或算子的何种假设下，稳定外推是可能的，并且可以外推到超出训练域多远？现有理论通过加性惩罚项控制测试误差，这些惩罚项衡量训练分布与测试分布之间的差异。此类保证显示了对小分布偏移的鲁棒性，但与经验观察到的OOD性能相比可能过于悲观。我们识别出一类全纯函数和算子，即使存在大的分布偏移，其OOD泛化误差也能以代数速率收敛。这一现象源于更高索引坐标的平滑性增加，导致我们称之为“高维度的祝福”。对于使用多项式、深度神经网络或深度神经算子进行学习，我们为支持在适当域上的任意测试测度推导出显式速率，并量化可容许域如何依赖于函数或算子的底层正则性。我们的外推保证独立于测试分布，仅依赖于其支撑集。我们还提出了一系列跨多种函数和算子的数值实验，以支持主要理论发现。

英文摘要

Out-of-distribution (OOD) generalization is a central challenge in scientific machine learning. We study regression problems in which the test distribution differs from the training distribution and ask: under what assumptions on the target function or operator is stable extrapolation possible, and how far beyond the training domain can one extrapolate? Existing theory controls the test error through additive penalties measuring the discrepancy between the training and test distributions. Such guarantees show robustness to small distribution shifts, but can be very pessimistic in comparison to OOD performance observed empirically. We identify classes of holomorphic functions and operators for which the OOD generalization error converges at algebraic rates even in the presence of large distribution shifts. This phenomenon stems from the increasing smoothness of higher-index coordinates, leading to what we term a `blessing of high dimensionality'. For learning with either polynomials, deep neural networks or deep neural operators, we derive explicit rates for arbitrary test measures supported on suitable domains and quantify how the admissible domain depends on the underlying regularity of the function or operator. Our extrapolation guarantees are independent of the test distribution, depending only on its support. We also present a series of numerical experiments across a range of functions and operators that support the main theoretical findings.

发表机构

• Simon Fraser University（西蒙弗雷泽大学）
• Concordia University（康考迪亚大学）

机构由 AI 辅助整理，请以论文原文为准。