Memorizing Gaussians with no over-parameterizaion via gradient decent on neural networks
Hebrew University（希伯来大学）；Google（谷歌）
英文摘要：
We prove that a single step of gradient decent over depth two network, with $q$ hidden neurons, starting from orthogonal initialization, can memorize $Ω\left(\frac{dq}{\log^4(d)}\right)$ independent and randomly labeled Gaussians in $\mathbb{R}^d$. The result is valid for a large class of activation functions, which includes the absolute value.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
