重新思考数据表示中具有不变性的深度学习
Rethink Deep Learning with Invariance in Data Representation
AI总结:
本文从历史视角探讨数据表示中的不变性原则,指出其在深度学习前后的发展、被忽视后的瓶颈及回归形成的几何深度学习领域,并识别相关研究困境、前景工作等。
摘要:
将不变性整合到数据表示中是智能系统和网络应用的原则性设计。表示起着基础性作用,系统和应用都建立在数字输入的有意义表示(而非原始数据)之上。事实上,此类表示的恰当设计/学习依赖于与目标任务相关的先验知识。在此,来自埃尔朗根纲领的对称性概念可能是最富有成效的先验——通俗地说,系统的对称性是一种使系统的某种属性保持不变的变换。对称性先验无处不在,例如平移作为物体分类的对称性,其中物体类别在平移下保持不变。对不变性的探索与模式识别和数据挖掘本身一样古老。在深度学习时代之前,不变性设计一直是各种表示(如SIFT)的基石。当我们进入深度学习早期时代时,不变性原则在很大程度上被忽视,取而代之的是数据驱动范式(如CNN)。然而,这种忽视并未持续太久,他们很快就遇到了关于鲁棒性、可解释性、效率等方面的瓶颈。在重新思考深度学习的时代,不变性原则回归,形成了一个被称为几何深度学习(GDL)的新领域。在本教程中,我们将给出数据表示中不变性的历史视角。更重要的是,我们将识别这些研究困境、有前景的工作、未来方向和网络应用。
英文摘要:
Integrating invariance into data representations is a principled design in intelligent systems and web applications. Representations play a fundamental role, where systems and applications are both built on meaningful representations of digital inputs (rather than the raw data). In fact, the proper design/learning of such representations relies on priors w.r.t. the task of interest. Here, the concept of symmetry from the Erlangen Program may be the most fruitful prior -- informally, a symmetry of a system is a transformation that leaves a certain property of the system invariant. Symmetry priors are ubiquitous, e.g., translation as a symmetry of the object classification, where object category is invariant under translation. The quest for invariance is as old as pattern recognition and data mining itself. Invariant design has been the cornerstone of various representations in the era before deep learning, such as the SIFT. As we enter the early era of deep learning, the invariance principle is largely ignored and replaced by a data-driven paradigm, such as the CNN. However, this neglect did not last long before they encountered bottlenecks regarding robustness, interpretability, efficiency, and so on. The invariance principle has returned in the era of rethinking deep learning, forming a new field known as Geometric Deep Learning (GDL). In this tutorial, we will give a historical perspective of the invariance in data representations. More importantly, we will identify those research dilemmas, promising works, future directions, and web applications.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。