arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2302.08210cs.LG
收藏

深度学习的几何优化综述:从欧几里得空间到黎曼流形

A Survey of Geometric Optimization for Deep Learning: From Euclidean Space to Riemannian Manifold

East China Normal University(华东师范大学)

Yanhong Fei, Xian Wei, Yingjie Liu, Zhengyu Li, Mingsong Chen

摘要:

尽管深度学习(DL)在复杂的人工智能(AI)任务中取得了成功,但它存在各种臭名昭著的问题(例如,特征冗余和消失或爆炸梯度),因为更新欧几里德空间中的参数不能完全利用解空间的几何结构。作为一种有前途的替代解决方案,基于黎曼的DL使用几何优化来更新黎曼流形上的参数,并可以利用底层的几何信息。因此,本文提出了一个全面的调查应用几何优化DL。本文首先介绍了几何优化的基本过程,包括各种几何优化器和黎曼流形的一些概念。随后,本文研究了几何优化在不同DL网络中的应用,例如,卷积神经网络、递归神经网络、迁移学习和最优传输。此外,还讨论了实现流形优化的典型公共工具箱。最后,本文对不同深度几何优化方法在图像识别场景下的性能进行了比较。

英文摘要:

Although Deep Learning (DL) has achieved success in complex Artificial Intelligence (AI) tasks, it suffers from various notorious problems (e.g., feature redundancy, and vanishing or exploding gradients), since updating parameters in Euclidean space cannot fully exploit the geometric structure of the solution space. As a promising alternative solution, Riemannian-based DL uses geometric optimization to update parameters on Riemannian manifolds and can leverage the underlying geometric information. Accordingly, this article presents a comprehensive survey of applying geometric optimization in DL. At first, this article introduces the basic procedure of the geometric optimization, including various geometric optimizers and some concepts of Riemannian manifold. Subsequently, this article investigates the application of geometric optimization in different DL networks in various AI tasks, e.g., convolution neural network, recurrent neural network, transfer learning, and optimal transport. Additionally, typical public toolboxes that implement optimization on manifold are also discussed. Finally, this article makes a performance comparison between different deep geometric optimization methods under image recognition scenarios.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑