草图岭回归：优化视角、统计视角和模型平均
Sketched Ridge Regression: Optimization Perspective, Statistical Perspective, and Model Averaging
AI总结：
本文从优化和统计角度研究了草图和Hessian草图在矩阵岭回归中的影响，发现经典草图能近似最优解，而Hessian草图则不同。通过理论和实验表明，模型平均可显著降低真实与草图解间的风险差距。
摘要：
我们探讨了经典草图和Hessian草图在近似求解矩阵岭回归（MRR）问题中的统计和优化影响。先前研究量化了经典草图对更简单的最小二乘回归（LSR）问题的影响。我们证明经典草图对MRR的优化属性的影响与对LSR的影响类似：即恢复近似最优解。相反，Hessian草图没有这种保证，其近似误差由响应中的“质量”与最优目标值之间的微妙交互决定。对于两种类型的近似，sketched MRR中的正则化导致与sketched LSR不同的统计特性。特别是，在sketched MRR中存在偏误-方差权衡，这在sketched LSR中不存在。我们提供了sketched MRR的偏误和方差的上界和下界，这些界限表明经典草图显著增加方差，而Hessian草图显著增加偏误。经验上，sketched MRR的解的风险可能比最优MRR解高一个数量级。我们理论和实证表明，模型平均显著降低真实解与sketched解风险之间的差距。因此，在并行或分布式设置中，草图结合模型平均是一种强大的技术，能够快速获得近似最优解，同时大幅减轻草图带来的统计风险增加。
英文摘要：
We address the statistical and optimization impacts of the classical sketch and Hessian sketch used to approximately solve the Matrix Ridge Regression (MRR) problem. Prior research has quantified the effects of classical sketch on the strictly simpler least squares regression (LSR) problem. We establish that classical sketch has a similar effect upon the optimization properties of MRR as it does on those of LSR: namely, it recovers nearly optimal solutions. By contrast, Hessian sketch does not have this guarantee, instead, the approximation error is governed by a subtle interplay between the "mass" in the responses and the optimal objective value. For both types of approximation, the regularization in the sketched MRR problem results in significantly different statistical properties from those of the sketched LSR problem. In particular, there is a bias-variance trade-off in sketched MRR that is not present in sketched LSR. We provide upper and lower bounds on the bias and variance of sketched MRR, these bounds show that classical sketch significantly increases the variance, while Hessian sketch significantly increases the bias. Empirically, sketched MRR solutions can have risks that are higher by an order-of-magnitude than those of the optimal MRR solutions. We establish theoretically and empirically that model averaging greatly decreases the gap between the risks of the true and sketched solutions to the MRR problem. Thus, in parallel or distributed settings, sketching combined with model averaging is a powerful technique that quickly obtains near-optimal solutions to the MRR problem while greatly mitigating the increased statistical risk incurred by sketching.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
