神经网络可以两次学习同一个模型吗？从决策边界的角度考察再生产和双下降
Can Neural Nets Learn the Same Model Twice? Investigating Reproducibility and Double Descent from the Decision Boundary Perspective
摘要：
我们讨论了可视化神经网络决策边界和决策区域的方法。我们使用这些可视化来研究与神经网络训练中的可重复性和泛化相关的问题。我们观察到，模型架构（及其相关的归纳偏差）的变化会导致决策边界的明显变化，而相同架构的多次运行会产生具有很强相似性的结果，特别是在宽架构的情况下。我们还使用决策边界方法来可视化双下降现象。我们看到，决策边界再现性强烈依赖于模型的宽度。在插值阈值附近，神经网络的决策边界被分割成许多小的决策区域，并且这些区域是不可再现的。同时，非常窄和非常宽的网络在它们的决策边界中具有高水平的可再现性，而决策区域相对较少。我们讨论了我们的观察与凸模型中的双下降现象理论的关系。代码可在https://github.com/somepago/dbViz上获得
英文摘要：
We discuss methods for visualizing neural network decision boundaries and decision regions. We use these visualizations to investigate issues related to reproducibility and generalization in neural network training. We observe that changes in model architecture (and its associate inductive bias) cause visible changes in decision boundaries, while multiple runs with the same architecture yield results with strong similarities, especially in the case of wide architectures. We also use decision boundary methods to visualize double descent phenomena. We see that decision boundary reproducibility depends strongly on model width. Near the threshold of interpolation, neural network decision boundaries become fragmented into many small decision regions, and these regions are non-reproducible. Meanwhile, very narrows and very wide networks have high levels of reproducibility in their decision boundaries with relatively few decision regions. We discuss how our observations relate to the theory of double descent phenomena in convex models. Code is available at https://github.com/somepago/dbViz
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
