在线广义稀疏回归：过参数化如何发挥作用？
Online Generalized Sparse Regression: How Does Overparametrization Help?
- University of Toronto（多伦多大学）
- MBZUAI（穆罕默德·本·扎耶德人工智能大学）
机构由 AI 辅助整理，请以论文原文为准。
AI总结：
本文针对在线广义稀疏回归的四大挑战，提出在线广义稀疏约束回归框架，设计高效在线硬阈值算法，在适当过参数化下实现最优统计速率的全局收敛，数值实验显示其性能优于现有最优方法。
AI中文摘要：
正则化稀疏回归在离线场景中已得到广泛研究，但在线形式仍相对未被充分探索。这一空白源于四大关键挑战：（i）无法在每一轮在线迭代中动态更新正则化参数；（ii）存储与内存复杂度的管理；（iii）通过闭式更新实现实时计算，而非每轮求解完整优化问题；（iv）在现实假设下达到最优统计保证。本文提出一种在线广义稀疏约束回归框架，聚焦于在线基数约束线性回归与低秩矩阵感知。与在线正则化回归不同，该约束形式无需动态参数调优。我们引入一种高效的在线硬阈值算法，该算法执行闭式更新且仅需存储统计摘要，使其在计算、内存与存储方面均具效率。尽管该形式存在固有非凸性与组合性质，但在现实假设下，只要投影集被适当过参数化，我们的算法即可达到最优统计速率下的全局收敛。数值实验表明，我们的方法始终优于现有最优替代方案。
英文摘要：
Regularized sparse regression has been extensively studied in the offline setting, but online formulation remains relatively under-explored. This gap stems from four key challenges: (i) the infeasibility of dynamically updating the regularization parameter in every online round, (ii) managing storage and memory complexity, (iii) enabling real-time computation via closed-form updates rather than solving full optimization problems at each round, and (iv) achieving optimal statistical guarantees under realistic assumptions. In this paper, we propose an online generalized-sparsity-constrained regression framework, focusing on online cardinality-constrained linear regression and low-rank matrix sensing. Unlike online regularized regression, our constrained formulation eliminates the need for dynamic parameter tuning. We introduce an efficient online hard-thresholding algorithm that performs closed-form updates and requires storing only summary statistics, making it computationally, memory, and storage efficient. Despite the inherent nonconvexity and combinatorial nature of the formulation, our algorithm achieves global convergence at the optimal statistical rate under realistic assumptions, provided that the projection set is properly overparameterized. Numerical experiments demonstrate that our method consistently outperforms state-of-the-art alternatives.
