通过折扣VAW和子空间逼近在RKHS中研究在线回归的动态遗憾
Dynamic Regret for Online Regression in RKHS via Discounted VAW and Subspace Approximation
- Institute of Mathematics, Mechanics and Computer Sciences of the Southern Federal University（南方联邦大学数学、力学与计算机科学研究所）
机构由 AI 辅助整理，请以论文原文为准。
AI总结：
本文提出在RKHS中利用折扣VAW和子空间逼近方法，研究在线回归的动态遗憾问题，通过有限维子空间近似将有限维折扣VAW方法扩展到RKHS，并通过正交截断方法实现快速和慢速情形下的动态遗憾界。
AI中文摘要：
我们研究在再生核希尔伯特空间中使用平方损失的在线回归问题，基于动态遗憾准则。学习者与一个随时间变化的比较序列进行比较，其界限依赖于其在RKHS范数下的路径长度。所提出的方法通过有限维子空间近似将Jacobsen & Cutkosky (2024)的有限维折扣Vovk--Azoury--Warmuth方法扩展到RKHS设置。对于固定的子空间，我们运行一个基于VAW的装入体，其在折扣因子的几何网格上运行。额外的近似误差通过核截面的均匀投影误差进行控制。然后我们引入了一种通用的正交截断方法：从核的特征展开开始，通过引入一个内积使特征函数正交，然后使用前几个基函数的跨度作为有限维近似空间。所得到的子空间减少应用于多种近似方案。显式的特征展开为高斯和解析点积核提供了快速模式的界限。Mercer截断提供了一种谱近似方法，并根据特征值衰减情况导致快速和慢速模式下的动态遗憾界。最后，我们研究了由核截面张成的子空间，并将此构造应用于Matérn核。
英文摘要：
We study online regression with the square loss in a reproducing kernel Hilbert space under a dynamic regret criterion. The learner is compared with a time-varying comparator sequence, and the bounds depend on its path length in the RKHS norm. The proposed method transfers the finite-dimensional discounted Vovk--Azoury--Warmuth approach of Jacobsen \& Cutkosky (2024) to the RKHS setting by means of finite-dimensional subspace approximations. For a fixed subspace, we run a VAW-based ensemble of discounted VAW forecasters over a geometric grid of discount factors. The additional approximation error is controlled by the uniform projection error of kernel sections. We then introduce a general orthogonal truncation method: starting from a feature expansion of the kernel, we construct the associated RKHS by introducing an inner product that makes the feature functions orthonormal, and then use the spans of the first basis functions as finite-dimensional approximation spaces. The resulting subspace reduction is applied to several approximation schemes. Explicit feature expansions yield fast-regime bounds for Gaussian and analytic dot-product kernels. Mercer truncations provide a spectral approximation method and lead to dynamic regret bounds in fast and slow regimes, depending on the eigenvalue decay. Finally, we study subspaces spanned by kernel sections and apply this construction to Matérn kernels.
