Complex Dynamics in Simple Neural Networks: Understanding Gradient Flow in Phase Retrieval
Université Paris-Saclay（巴黎萨克雷大学）；CNRS（法国国家科学研究中心）；CEA（法国原子能和替代能源委员会）；Institut de physique théorique（理论物理研究所）；Laboratoire de Physique de l’Ecole normale supérieure ENS（巴黎高等师范学院物理实验室）；Université PSL（巴黎文理研究大学）；Sorbonne Université（索邦大学）；Université Paris-Diderot（巴黎第七大学）；Sorbonne Paris Cité（巴黎索邦西岱大学）；King’s College London（伦敦国王学院）
Despite the widespread use of gradient-based algorithms for optimizing high-dimensional non-convex functions, understanding their ability of finding good minima instead of being trapped in spurious ones remains to a large extent an open problem. Here we focus on gradient flow dynamics for phase retrieval from random measurements. When the ratio of the number of measurements over the input dimension is small the dynamics remains trapped in spurious minima with large basins of attraction. We find analytically that above a critical ratio those critical points become unstable developing a negative direction toward the signal. By numerical experiments we show that in this regime the gradient flow algorithm is not trapped; it drifts away from the spurious critical points along the unstable direction and succeeds in finding the global minimum. Using tools from statistical physics we characterize this phenomenon, which is related to a BBP-type transition in the Hessian of the spurious minima.
Gradient Descent with Random Initialization: Fast Global Convergence for Nonconvex Phase Retrieval
Department of Electrical Engineering, Princeton University（普林斯顿大学电气工程系）；Department of Electrical and Computer Engineering, Carnegie Mellon University（卡内基梅隆大学电气与计算机工程系）；Department of Operations Research and Financial Engineering, Princeton University（普林斯顿大学运筹学与金融工程系）
本文研究了通过二次方程恢复目标对象的问题，证明了在高斯设计下，随机初始化的梯度下降能在O(log n + log(1/ε))次迭代中获得ε精度的解，从而实现了计算和样本复杂度的近最优性，为相位恢复提供了首个无需精心设计初始化、样本分割或复杂鞍点逃离方案的全局收敛保证。
On the conditional equivalence of phase retrieval algorithms
Fakultät für Physik, Ludwig-Maximilian-Universität München（物理系，路德维希-马克西米利安慕尼黑大学）；Munich Center for Machine Learning (MCML)（慕尼黑机器学习中心）；Max Planck Institut für Quantenoptik（马克斯·普朗克量子光学研究所）
