arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2104.13478cs.LGcs.AIcs.CGcs.CVstat.ML
收藏

几何深度学习:网格、组、图、测地线和量规

Geometric Deep Learning: Grids, Groups, Graphs, Geodesics, and Gauges

Imperial College London(伦敦帝国学院);USI IDSIA(瑞士意大利语区人工智能研究所);Twitter(推特);New York University(纽约大学);Qualcomm AI Research(高通AI研究院);Qualcomm Technologies, Inc.(高通技术公司);DeepMind(深度思维)

Michael M. Bronstein, Joan Bruna, Taco Cohen, Petar Veličković

摘要:

过去十年见证了数据科学和机器学习的实验性革命,其中以深度学习方法为代表。事实上,许多以前认为遥不可及的高维学习任务--如计算机视觉、下围棋或蛋白质折叠--实际上在适当的计算规模下是可行的。值得注意的是,深度学习的本质是建立在两个简单的算法原则上的:第一,表示或特征学习的概念,即适应的,通常是分层的,特征捕获每个任务的适当规律性概念,第二,通过局部梯度下降类型的方法进行学习,通常以反向传播的方式实现。虽然在高维中学习通用函数是一个被诅咒的估计问题,但大多数感兴趣的任务都不是通用的,并且具有基本的预定义的,由物理世界的底层低维和结构引起的复杂性。本文关注的是通过统一的几何原则,可以在整个应用范围广泛的应用程序暴露这些问题。这种“几何统一”的努力,本着Felix Klein的Erlangen计划的精神,具有双重目的:一方面,它提供了一个通用的数学框架来研究最成功的神经网络架构,如CNN,RNN,GNN和Transformers。另一方面,它给出了一个建设性的程序,将先前的物理知识纳入神经架构,并提供了原则性的方式来构建未来的架构尚未发明。

英文摘要:

The last decade has witnessed an experimental revolution in data science and machine learning, epitomised by deep learning methods. Indeed, many high-dimensional learning tasks previously thought to be beyond reach -- such as computer vision, playing Go, or protein folding -- are in fact feasible with appropriate computational scale. Remarkably, the essence of deep learning is built from two simple algorithmic principles: first, the notion of representation or feature learning, whereby adapted, often hierarchical, features capture the appropriate notion of regularity for each task, and second, learning by local gradient-descent type methods, typically implemented as backpropagation. While learning generic functions in high dimensions is a cursed estimation problem, most tasks of interest are not generic, and come with essential pre-defined regularities arising from the underlying low-dimensionality and structure of the physical world. This text is concerned with exposing these regularities through unified geometric principles that can be applied throughout a wide spectrum of applications. Such a 'geometric unification' endeavour, in the spirit of Felix Klein's Erlangen Program, serves a dual purpose: on one hand, it provides a common mathematical framework to study the most successful neural network architectures, such as CNNs, RNNs, GNNs, and Transformers. On the other hand, it gives a constructive procedure to incorporate prior physical knowledge into neural architectures and provide principled way to build future architectures yet to be invented.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑