arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2410.02680stat.MLcs.LG
收藏

高度自适应岭回归

Highly Adaptive Ridge

University of California, Berkeley(加州大学伯克利分校);Division of Biostatistics(生物统计系)

Alejandro Schuler, Mingxun Wang, Alexander Hagemeister, Mark van der Laan

AI总结:

本文提出高度自适应岭回归(HAR),通过基于饱和零阶张量积样条基展开的数据自适应核实现核岭回归,在右连续函数类上达到维度无关的 $n^{-1/3}$ L2 收敛速率,并在小数据集上表现优于现有算法。

摘要:

在本文中,我们提出了高度自适应岭回归(Highly Adaptive Ridge,HAR):一种在具有平方可积截面导数的右连续函数类中实现 $n^{-1/3}$ 维度无关 L2 收敛速率的回归方法。这是一个大型非参数函数类,特别适用于表格数据。HAR 本质上是核岭回归,使用基于饱和零阶张量积样条基展开的特定数据自适应核。我们使用模拟和真实数据来验证我们的理论。我们证明了经验性能优于最先进的算法,尤其是在小数据集上。

英文摘要:

In this paper we propose the Highly Adaptive Ridge (HAR): a regression method that achieves an $n^{-1/3}$ dimension-free $\mathscr L_2$ convergence rate (up to log factors) in the class of right-continuous functions with square-integrable sectional derivatives. This is a large nonparametric function class that is particularly appropriate for tabular data. For squared error loss, HAR is exactly kernel ridge regression with a specific data-adaptive kernel based on a saturated zero-order tensor-product spline basis expansion, and thus scales reasonably well, computationally. We prove close connections between HAR and kernel ridge regression in Sobolev spaces of dominating mixed derivatives, and highlight practical differences. We use simulation and real data to confirm our theory.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑