arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2005.11603cs.NEcs.LG
收藏

神经网络中恢复力预测和损伤恢复的几何算法

Geometric algorithms for predicting resilience and recovering damage in neural networks

Caltech(加州理工学院);UCLA(加州大学洛杉矶分校)

Guruprasad Raghavan, Jiayi Li, Matt Thomson

摘要:

生物神经网络已经进化到在电路严重损坏的情况下仍能保持性能。为了在损伤中幸存下来,生物网络架构既具有对组件丢失的内在弹性,又激活恢复程序,通过可塑性调整网络权重以稳定性能。尽管弹性在技术应用中的重要性,但人工神经网络的弹性却知之甚少,自主恢复算法也有待开发。在本文中,我们建立了一个数学框架来分析人工神经网络的弹性通过微分几何的镜头。我们的几何语言提供了自然算法,可以识别训练网络中的局部漏洞,以及动态调整网络以补偿损坏的恢复算法。我们揭示了常用图像分析网络中的显著漏洞,例如分别在MNIST和CIFAR10上训练的MLP和CNN。我们还发现了高性能的恢复路径,使相同的网络能够动态地重新调整其参数以补偿损坏。从广义上讲,我们的工作提供了赋予人工系统弹性和快速恢复例程的程序,以增强其与物联网设备的集成,并使其能够部署关键应用程序。

英文摘要:

Biological neural networks have evolved to maintain performance despite significant circuit damage. To survive damage, biological network architectures have both intrinsic resilience to component loss and also activate recovery programs that adjust network weights through plasticity to stabilize performance. Despite the importance of resilience in technology applications, the resilience of artificial neural networks is poorly understood, and autonomous recovery algorithms have yet to be developed. In this paper, we establish a mathematical framework to analyze the resilience of artificial neural networks through the lens of differential geometry. Our geometric language provides natural algorithms that identify local vulnerabilities in trained networks as well as recovery algorithms that dynamically adjust networks to compensate for damage. We reveal striking vulnerabilities in commonly used image analysis networks, like MLP's and CNN's trained on MNIST and CIFAR10 respectively. We also uncover high-performance recovery paths that enable the same networks to dynamically re-adjust their parameters to compensate for damage. Broadly, our work provides procedures that endow artificial systems with resilience and rapid-recovery routines to enhance their integration with IoT devices as well as enable their deployment for critical applications.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑