arXivDaily arXiv每日学术速递 周一至周五更新
北美退伍军人节(Veterans Day),12 日无更新
arXiv 2408.13591stat.MLcs.LG
收藏

基于随机特征的最优核分位数学习

Optimal Kernel Quantile Learning with Random Features

School of Statistics(统计学院);Shanghai University of Finance and Economics(上海财经大学)

Caixing Wang, Xingdong Feng

AI总结:

本文研究带随机特征的核分位数回归,通过精细误差分解及其与 KRR-RF 的联系,建立容量依赖的极小极大最优学习率,并扩展至不可知设定与 Lipschitz 损失。

摘要:

随机特征(RF)方法是用于可扩展核方法的一种成熟且高效的工具,但现有文献主要关注带随机特征的核岭回归(KRR-RF),该方法在处理具有重尾噪声的异质数据时存在局限。本文对带随机特征的核分位数回归(KQR-RF)进行了泛化研究,通过引入精细的误差分解并建立 KQR-RF 与 KRR-RF 之间的新联系,解释了 KQR-RF 中 check loss 的非光滑性。我们的研究在关于随机特征数量的温和条件下,建立了 KQR-RF 依赖容量的学习速率,这些速率在至多若干对数因子的意义下是极小极大最优的。重要的是,借助数据依赖的采样策略,我们的理论结果可以扩展到不可知设定,即目标分位数函数可能并不精确属于所假设的核空间。通过略微修改假设,依赖容量的误差分析也可应用于具有 Lipschitz 连续损失的情形,从而在机器学习社区中实现更广泛的应用。为验证理论发现,本文开展了模拟实验和真实数据应用。

英文摘要:

The random feature (RF) approach is a well-established and efficient tool for scalable kernel methods, but existing literature has primarily focused on kernel ridge regression with random features (KRR-RF), which has limitations in handling heterogeneous data with heavy-tailed noises. This paper presents a generalization study of kernel quantile regression with random features (KQR-RF), which accounts for the non-smoothness of the check loss in KQR-RF by introducing a refined error decomposition and establishing a novel connection between KQR-RF and KRR-RF. Our study establishes the capacity-dependent learning rates for KQR-RF under mild conditions on the number of RFs, which are minimax optimal up to some logarithmic factors. Importantly, our theoretical results, utilizing a data-dependent sampling strategy, can be extended to cover the agnostic setting where the target quantile function may not precisely align with the assumed kernel space. By slightly modifying our assumptions, the capacity-dependent error analysis can also be applied to cases with Lipschitz continuous losses, enabling broader applications in the machine learning community. To validate our theoretical findings, simulated experiments and a real data application are conducted.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑