基于轨迹预测的超参数调优：矩阵感知中的随机近似线性方法
Hyperparameter tuning via trajectory predictions: Stochastic prox-linear methods in matrix sensing
AI总结：
针对非凸优化随机算法的超参数鲁棒性问题，研究秩1矩阵感知的小批量近似线性算法，推导可预测误差的确定性递推，证明其非渐近准确性，并实现无需运行算法的超参数调优。
摘要：
受理解对超参数选择具有鲁棒性的非凸优化随机算法的需求驱动，我们分析了一种小批量近似线性迭代算法，用于从受噪声干扰的秩1高斯测量值中恢复未知的秩1矩阵。我们推导出一种预测该方法误差的确定性递推关系，并使用非渐近框架证明，该预测对于任意批量大小和较大范围的步长均准确。特别地，我们的分析表明，该方法尽管具有随机性，但能以固定步长从局部初始化线性收敛到统计误差下限。我们的分析还揭示了批量大小、步长和噪声水平如何影响（线性）收敛速率以及最终的统计估计误差，并且我们展示了如何利用我们的确定性预测来执行超参数调优（例如步长和批量大小选择），而无需实际运行该方法。在技术层面，我们的分析部分得益于证明了经验迭代值围绕我们确定性预测的波动幅度与前一次迭代的误差成正比。
英文摘要：
Motivated by the desire to understand stochastic algorithms for nonconvex optimization that are robust to their hyperparameter choices, we analyze a mini-batched prox-linear iterative algorithm for the problem of recovering an unknown rank-1 matrix from rank-1 Gaussian measurements corrupted by noise. We derive a deterministic recursion that predicts the error of this method and show, using a non-asymptotic framework, that this prediction is accurate for any batch-size and a large range of step-sizes. In particular, our analysis reveals that this method, though stochastic, converges linearly from a local initialization with a fixed step-size to a statistical error floor. Our analysis also exposes how the batch-size, step-size, and noise level affect the (linear) convergence rate and the eventual statistical estimation error, and we demonstrate how to use our deterministic predictions to perform hyperparameter tuning (e.g. step-size and batch-size selection) without ever running the method. On a technical level, our analysis is enabled in part by showing that the fluctuations of the empirical iterates around our deterministic predictions scale with the error of the previous iterate.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
