arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2104.14372cs.LGcs.AI
收藏

深度学习中规格不足的神经各向异性观点

A neural anisotropic view of underspecification in deep learning

Ecole Polytechnique Fédérale de Lausanne(洛桑联邦理工学院);Universidad Nacional de Ingeniería(国立工程大学);ETH Zürich(苏黎世联邦理工学院)

Guillermo Ortiz-Jimenez, Itamar Franco Salazar-Reque, Apostolos Modas, Seyed-Mohsen Moosavi-Dezfooli, Pascal Frossard

摘要:

大多数机器学习管道的规格不足意味着我们不能仅仅依靠验证性能来评估深度学习系统对自然发生的分布偏移的鲁棒性。相反,要确保神经网络可以在大量不同的情况下进行泛化,需要了解它解决任务的具体方式。在这项工作中,我们建议从几何的角度来研究这个问题,目的是了解两个关键特性的神经网络解决方案在欠指定的设置:如何学习函数的几何形状相关的数据表示?而且,深度网络是否总是偏向于更简单的解决方案,就像最近的文献中所描述的那样?我们发现,神经网络处理这些问题的方式是高度依赖于数据表示,影响的几何形状和复杂性的学习预测。我们的研究结果强调,理解深度学习中的架构归纳偏差对于解决这些系统的公平性、鲁棒性和泛化性至关重要。

英文摘要:

The underspecification of most machine learning pipelines means that we cannot rely solely on validation performance to assess the robustness of deep learning systems to naturally occurring distribution shifts. Instead, making sure that a neural network can generalize across a large number of different situations requires to understand the specific way in which it solves a task. In this work, we propose to study this problem from a geometric perspective with the aim to understand two key characteristics of neural network solutions in underspecified settings: how is the geometry of the learned function related to the data representation? And, are deep networks always biased towards simpler solutions, as conjectured in recent literature? We show that the way neural networks handle the underspecification of these problems is highly dependent on the data representation, affecting both the geometry and the complexity of the learned predictors. Our results highlight that understanding the architectural inductive bias in deep learning is fundamental to address the fairness, robustness, and generalization of these systems.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑