arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2008.00741cs.LGcs.NEstat.ML
收藏

权向量的低损失连接:基于分布的方法

Low-loss connection of weight vectors: distribution-based approaches

Skolkovo Institute of Science and Technology(斯科尔科沃科学技术研究院)

Ivan Anokhin, Dmitry Yarotsky

摘要:

最近的研究表明,超参数网络的损失面的子水平集是精确连通或近似连通的。我们描述和比较实验面板的方法,用于连接两个低损耗点的低损耗曲线在这个表面上。我们的方法在准确性和复杂性方面各不相同。我们的大多数方法是基于“宏观”的分布假设,有些是不敏感的详细属性的点连接。一些方法需要预先训练“全局连接模型”,然后可以将其应用于任何一对点。该方法的准确性通常与其复杂性和对端点细节的敏感性相关。

英文摘要:

Recent research shows that sublevel sets of the loss surfaces of overparameterized networks are connected, exactly or approximately. We describe and compare experimentally a panel of methods used to connect two low-loss points by a low-loss curve on this surface. Our methods vary in accuracy and complexity. Most of our methods are based on "macroscopic" distributional assumptions, and some are insensitive to the detailed properties of the points to be connected. Some methods require a prior training of a "global connection model" which can then be applied to any pair of points. The accuracy of the method generally correlates with its complexity and sensitivity to the endpoint detail.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑