Stochastic Online Linear Regression: the Forward Algorithm to Replace Ridge
- Univ. Lille（里尔大学）
- CNRS（法国国家科学研究中心）
- Inria（法国国家信息与自动化研究所）
- Centrale Lille（里尔中央理工学院）
- Criteo（Criteo公司）
- ENSAE（法国国立统计与经济管理学院）
- ENS PARIS-SACLAY（巴黎萨克雷高等师范学院）
机构由 AI 辅助整理，请以论文原文为准。
英文摘要：
We consider the problem of online linear regression in the stochastic setting. We derive high probability regret bounds for online ridge regression and the forward algorithm. This enables us to compare online regression algorithms more accurately and eliminate assumptions of bounded observations and predictions. Our study advocates for the use of the forward algorithm in lieu of ridge due to its enhanced bounds and robustness to the regularization parameter. Moreover, we explain how to integrate it in algorithms involving linear function approximation to remove a boundedness assumption without deteriorating theoretical bounds. We showcase this modification in linear bandit settings where it yields improved regret bounds. Last, we provide numerical experiments to illustrate our results and endorse our intuitions.
