发表机构
Purdue University; University of Wisconsin-Madison（普渡大学; 威斯康星大学麦迪逊分校）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
本文证明在迭代岭回归中，每次迭代使用新素描相比复用素描具有可证明的优势，并提出残差感知采样规则，实验显示显著加速收敛。
AI 中文摘要
在过去的25年里，素描和采样已成为加速大规模回归的广泛使用的工具。在迭代随机求解器中，一个基本的设计选择是是否在每一步重复使用相同的素描或抽取新的随机性。对于（欠约束的）迭代岭回归与列采样，新素描是否具有可证明的优势仍然是一个开放问题：我们证明它们确实具有优势。新素描使我们能够仅沿当前残差解分析误差，而不是在整个Gram矩阵上均匀分析。这种方向性视角为杠杆分数和岭杠杆分数采样提供了更尖锐的收敛保证，更重要的是，导致了残差感知的采样规则。通过最小化相关素描矩阵-向量乘积的方差，我们推导出一个预言分布和预言分布的实用近似，包括一个具有（稍弱的）收敛保证的混合采样分布。在合成和真实数据上的实验，包括在Qwen2.5表示上的岭探针，支持我们的理论，显示出显著更快的收敛。
英文摘要
Over the past 25 years, sketching and sampling have become widely used tools for accelerating large-scale regression. In iterative randomized solvers, a basic design choice is whether to $\textit{reuse}$ the same sketch or draw $\textit{fresh}$ randomness at every step. For (under-constrained) iterative ridge regression with column sampling, whether fresh sketches offer provable advantages has remained open: $\textit{We show that they do.}$ Fresh sketching lets us analyze error only along the current residual solution, rather than uniformly over the entire Gram matrix. This directional view yields sharper convergence guarantees for leverage score and ridge leverage score sampling and, more importantly, leads to residual-aware sampling rules. By minimizing the variance of the relevant sketched matrix-vector product, we derive an oracle distribution and practical approximations to the oracle distribution, including a mixture sampling distribution with (somewhat weaker) convergence guarantees. Experiments on synthetic and real data, including ridge probes on Qwen2.5 representations, support our theory, showing substantially faster convergence.
