通过遍历测地线使网络稀疏化
Sparsifying networks by traversing Geodesics
摘要:
神经网络的权空间和函数流形的几何形状对于"理解" ML的复杂性起着重要的作用。在本文中,我们试图解决ML中的某些开放问题,通过几何的镜头来观察它们,最终将其与这些空间中等价函数的点或路径的发现联系起来。我们提出了一个数学框架来评估测地线的功能空间,找到高性能的路径从密集的网络,其稀疏的对应。我们的结果是在CIFAR-10训练的VGG-11和MNIST训练的MLP上获得的。从广义上讲,我们证明了该框架是通用的,可以应用于各种各样的问题,从稀疏化,以减轻灾难性遗忘。
英文摘要:
The geometry of weight spaces and functional manifolds of neural networks play an important role towards 'understanding' the intricacies of ML. In this paper, we attempt to solve certain open questions in ML, by viewing them through the lens of geometry, ultimately relating it to the discovery of points or paths of equivalent function in these spaces. We propose a mathematical framework to evaluate geodesics in the functional space, to find high-performance paths from a dense network to its sparser counterpart. Our results are obtained on VGG-11 trained on CIFAR-10 and MLP's trained on MNIST. Broadly, we demonstrate that the framework is general, and can be applied to a wide variety of problems, ranging from sparsification to alleviating catastrophic forgetting.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。