We study a spectral initialization method that serves a key role in recent work on estimating signals in nonconvex settings. Previous analysis of this method focuses on the phase retrieval problem and provides only performance bounds. In this paper, we consider arbitrary generalized linear sensing models and present a precise asymptotic characterization of the performance of the method in the high-dimensional limit. Our analysis also reveals a phase transition phenomenon that depends on the ratio between the number of samples and the signal dimension. When the ratio is below a minimum threshold, the estimates given by the spectral method are no better than random guesses drawn from a uniform distribution on the hypersphere, thus carrying no information; above a maximum threshold, the estimates become increasingly aligned with the target signal. The computational complexity of the method, as measured by the spectral gap, is also markedly different in the two phases. Worked examples and numerical results are provided to illustrate and verify the analytical predictions. In particular, simulations show that our asymptotic formulas provide accurate predictions for the actual performance of the spectral method even at moderate signal dimensions.
Gradient Descent with Random Initialization: Fast Global Convergence for Nonconvex Phase Retrieval
Department of Electrical Engineering, Princeton University(普林斯顿大学电气工程系);Department of Electrical and Computer Engineering, Carnegie Mellon University(卡内基梅隆大学电气与计算机工程系);Department of Operations Research and Financial Engineering, Princeton University(普林斯顿大学运筹学与金融工程系)
本文研究了通过二次方程恢复目标对象的问题,证明了在高斯设计下,随机初始化的梯度下降能在O(log n + log(1/ε))次迭代中获得ε精度的解,从而实现了计算和样本复杂度的近最优性,为相位恢复提供了首个无需精心设计初始化、样本分割或复杂鞍点逃离方案的全局收敛保证。
Small random initialization is akin to spectral learning: Optimization and generalization guarantees for overparameterized low-rank matrix reconstruction
Fast Signal Recovery from Saturated Measurements by Linear Loss and Nonconvex Penalties
Institute of Image Processing and Pattern Recognition, Shanghai Jiao Tong University(上海交通大学图像处理与模式识别研究所);University of Electronic Science and Technology of China(电子科技大学);Michigan State University(密歇根州立大学)