arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 1911.06756cond-mat.dis-nnstat.ML
收藏

对称二元感知器中解的聚类

Clustering of solutions in the symmetric binary perceptron

Carlo Baldassi, Riccardo Della Vecchia, Carlo Lucibello, Riccardo Zecchina

摘要:

神经网络模型的(非凸)损失景观的几何特征对于确保成功优化至关重要,最重要的是,具有良好的泛化能力。虽然极小化者的平坦性始终与良好的概括性相关,但很少有严格的工作来探索这种极小化者的存在条件,即使是在玩具模型中。在这里,我们考虑一个简单的神经网络模型,对称感知器,二进制权重。将学习问题表述为约束满足问题,类似的平坦极小化器变成了一个大而密集的解簇,而最小化器是孤立的解。我们执行的第一个步骤,严格证明了一个密集的集群的存在,在某些制度的参数,通过计算的第一和第二时刻的上限对任意接近的解决方案的存在。此外,我们提出了一个非严格的推导相同的界限为$y$解决方案在固定的成对距离。

英文摘要:

The geometrical features of the (non-convex) loss landscape of neural network models are crucial in ensuring successful optimization and, most importantly, the capability to generalize well. While minimizers' flatness consistently correlates with good generalization, there has been little rigorous work in exploring the condition of existence of such minimizers, even in toy models. Here we consider a simple neural network model, the symmetric perceptron, with binary weights. Phrasing the learning problem as a constraint satisfaction problem, the analogous of a flat minimizer becomes a large and dense cluster of solutions, while the narrowest minimizers are isolated solutions. We perform the first steps toward the rigorous proof of the existence of a dense cluster in certain regimes of the parameters, by computing the first and second moment upper bounds for the existence of pairs of arbitrarily close solutions. Moreover, we present a non rigorous derivation of the same bounds for sets of $y$ solutions at fixed pairwise distances.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑