深曲率套件
Deep Curvature Suite
摘要:
我们介绍了MLRG Deep Curvature套件,这是一个基于PyTorch的开源软件包,用于分析和可视化神经网络曲率和损失景观。尽管提供了丰富的信息到神经网络的属性和有用的各种设计任务,曲率信息仍然没有充分利用各种原因,我们的方法旨在弥合这一差距。我们提出了一个引物,包括其主要的实际需求和常见的误解,\textit{Lanczos algorithm},我们的包的理论骨干,并提出了一系列的例子的基础上合成玩具的例子和现实的现代神经网络测试CIFAR数据集,并显示我们的包对现有的竞争方法的优越性类似的目的。
英文摘要:
We present MLRG Deep Curvature suite, a PyTorch-based, open-source package for analysis and visualisation of neural network curvature and loss landscape. Despite of providing rich information into properties of neural network and useful for a various designed tasks, curvature information is still not made sufficient use for various reasons, and our method aims to bridge this gap. We present a primer, including its main practical desiderata and common misconceptions, of \textit{Lanczos algorithm}, the theoretical backbone of our package, and present a series of examples based on synthetic toy examples and realistic modern neural networks tested on CIFAR datasets, and show the superiority of our package against existing competing approaches for the similar purposes.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。