损失条形码：损失景观中逃逸性的拓扑度量
Loss Barcode: A Topological Measure of Escapability in Loss Landscapes
AI总结：
本文提出损失条形码作为评估损失景观逃逸性的拓扑度量，通过分析神经网络的拓扑障碍评分，揭示了深度学习中局部极小值的逃逸性与泛化误差的关系。
摘要：
神经网络训练通常基于随机梯度下降（SGD）。然而，鉴于损失函数的非凸性质和损失景观的复杂几何特征，对SGD收敛到良好局部极小值的能力的理解仍然有限。在本文中，我们应用拓扑数据分析方法来分析损失景观，以深入了解深度神经网络的学习过程和泛化特性。我们利用损失函数的拓扑性质，将梯度下降轨迹的局部行为与损失表面的全局性质联系起来。为此，我们借助鲁棒的拓扑不变量——损失函数的条形码，定义了神经网络的拓扑障碍评分（
英文摘要：
Neural network training is commonly based on SGD. However, the understanding of SGD's ability to converge to good local minima, given the non-convex nature of loss functions and the intricate geometric characteristics of loss landscapes, remains limited. In this paper, we apply topological data analysis methods to loss landscapes to gain insights into the learning process and generalization properties of deep neural networks. We use the loss function topology to relate the local behavior of gradient descent trajectories with the global properties of the loss surface. For this purpose, we define the neural network's Topological Obstructions score ("TO-score") with the help of robust topological invariants, barcodes of the loss function, which quantify the escapability of local minima for gradient-based optimization. Our two principal observations are: 1) the loss barcode of the neural network decreases with increasing depth and width, therefore the topological obstructions to learning diminish; 2) in certain situations there is a connection between the length of minima segments in the loss barcode and the minima's generalization errors. Our statements are based on extensive experiments with fully connected, convolutional, and transformer architectures and several datasets including MNIST, FMNIST, CIFAR10, CIFAR100, SVHN, and multilingual OSCAR text dataset.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
