发表机构
TOYOTA TECHNOLOGICAL INSTITUTE AT CHICAGO（芝加哥丰田理工学院）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
本论文针对神经网络研究中对其解决方案机制认知不足的问题，阐明并利用损失地形中的模式连通性特殊结构，以推动神经网络的可解释性研究。
AI 中文摘要
在过去十年中，神经网络已从学术上的好奇事物发展到推动国家市场的重要力量。尽管其研究和部署都取得了爆炸式增长，但人们对它们如何获得所做的解决方案却知之甚少，这既具有科学相关性，也对社会至关重要。当神经网络在自动驾驶、建筑、法律、招聘和医疗等领域做出决策时，已经出现并将继续出现意外后果。然而，试图对最大、最重要的生产系统的故障进行概括是一项非常困难的任务。不过，这些故障的迹象存在于神经网络的所有规模中，因此我们应该能够研究一个更易处理的场景。所有神经网络都必须经过称为训练的优化过程才能发挥作用。在很大程度上，理解神经网络就是理解它们的优化：它们通过什么过程以及接触哪些数据得出了结果。然而，该领域对这一主题的知识相当不精确。特别是，一种称为模式连通性（即能够在损失地形中连接神经网络）的奇特现象完全无法解释。本论文阐明、解释并利用了损失地形中的这种特殊结构。
英文摘要
Over the course of the last decade, neural networks have grown from an academic curiosity to moving the markets of nations. Despite this explosion in both research and deployment, relatively little is understood about how they achieve the solutions they do. This is both scientifically relevant, and pressing for society. When neural networks make decisions across self-driving, construction, law, hiring and health, there have been and will continue to be unintended consequences. However, attempting to generalize the failures of the largest and most important production systems makes for a very difficult task. Yet signs of these failures exist at all scales of neural networks, so we should be able to study a much more tractable setting. All neural networks must undergo an optimization process, called training, to be useful. To a great degree, understanding neural networks is understanding their optimization: through what process and exposure to which data did they arrive at their results. Yet our knowledge on this topic as a field is quite imprecise. In particular, a curious phenomenon called mode connectivity, the ability to connect neural networks in the loss surface, defies explanation entirely. This dissertation elucidates, explains and exploits this special structure in the loss landscape...
CommentsPhD dissertation. This article draws from arXiv:2408.11804
