arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2310.02850math.PRcond-mat.dis-nn
收藏

对称二元感知器的非典型解

On the Atypical Solutions of the Symmetric Binary Perceptron

Damien Barbier, Ahmed El Alaoui, Florent Krzakala, Lenka Zdeborová

AI总结:

该研究分析对称二元感知器中稀有高间隔解,发现其形成具有熵垒和能量垒的簇,并通过复制方法揭示复杂度-熵曲线的非凹性及Parisi参数发散时的平衡特性。

摘要:

我们研究随机二元对称感知器问题,重点关注稀有高间隔解的行为。虽然大多数解是孤立的,但我们证明这些稀有解属于具有广延熵的簇的一部分,启发式地对应于近似消息传递算法的非平凡不动点。我们通过局部熵(定义为Franz-Parisi势)来枚举这些簇,并在小约束密度$\alpha$(对应于消失间隔$\kappa$)的极限下,在熵浓度的一定假设下,利用第一和第二矩方法严格评估该局部熵。这一研究揭示了几个有趣的现象:i) 我们证明这些簇具有熵垒,即当$\kappa\le 1.429 \sqrt{-\alpha/\log\alpha}$时,熵作为距参考高间隔解距离的函数是非单调的,否则是单调的;并且它们具有能量垒,即当$\kappa\le 1.239 \sqrt{-\alpha/\log\alpha}$时,在距参考解中间距离处不存在解。间隔$\kappa$在$\sqrt{-\alpha/\log\alpha}$中的临界标度对应于Gamarnik等人(2022)早期关于重叠间隙性质的工作中得到的标度,该现象已知对某些高效算法构成障碍。ii) 我们利用复制方法证明,复杂度(此类解的簇数的对数)与熵(簇中解数的对数)曲线部分非凹,并对应于非常大的Parisi参数值,当Parisi参数发散时达到平衡。

英文摘要:

We study the random binary symmetric perceptron problem, focusing on the behavior of rare high-margin solutions. While most solutions are isolated, we demonstrate that these rare solutions are part of clusters of extensive entropy, heuristically corresponding to non-trivial fixed points of an approximate message-passing algorithm. We enumerate these clusters via a local entropy, defined as a Franz-Parisi potential, which we rigorously evaluate using the first and second moment methods in the limit of a small constraint density $α$ (corresponding to vanishing margin $κ$) under a certain assumption on the concentration of the entropy. This examination unveils several intriguing phenomena: i) We demonstrate that these clusters have an entropic barrier in the sense that the entropy as a function of the distance from the reference high-margin solution is non-monotone when $κ\le 1.429 \sqrt{-α/\logα}$, while it is monotone otherwise, and that they have an energetic barrier in the sense that there are no solutions at an intermediate distance from the reference solution when $κ\le 1.239 \sqrt{-α/ \logα}$. The critical scaling of the margin $κ$ in $\sqrt{-α/\logα}$ corresponds to the one obtained from the earlier work of Gamarnik et al. (2022) for the overlap-gap property, a phenomenon known to present a barrier to certain efficient algorithms. ii) We establish using the replica method that the complexity (the logarithm of the number of clusters of such solutions) versus entropy (the logarithm of the number of solutions in the clusters) curves are partly non-concave and correspond to very large values of the Parisi parameter, with the equilibrium being reached when the Parisi parameter diverges.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑