arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2203.04794math.OCcs.LG
收藏

流形上的几何优化及其在深度学习中的应用

Geometric Optimisation on Manifolds with Applications to Deep Learning

Keble College(基布尔学院)

Mario Lezcano-Casado

摘要:

我们设计并实现了一个Python库,以帮助非专家使用所有这些强大的工具,以一种高效、可扩展且简单的方式融入数据科学家、从业者和应用研究人员的工作流程。在这个库中实现的算法在设计时考虑了可用性和GPU效率,并且只需一行额外的代码就可以将它们添加到任何PyTorch模型中。我们展示了这些工具的有效性上的应用程序的流形上的优化设置的时间序列分析。在这种情况下,正交和酉优化用于约束和正则化经常性模型,避免消失和爆炸梯度问题。为GeoTorch设计的算法使我们能够在该系列模型的标准测试中获得最先进的结果。我们使用比较几何的工具来给出优化问题中感兴趣的数量的界限。特别是,我们建立在工作(考尔1976年),以明确的界限上的范数的二阶导数的黎曼指数。

英文摘要:

We design and implement a Python library to help the non-expert using all these powerful tools in a way that is efficient, extensible, and simple to incorporate into the workflow of the data scientist, practitioner, and applied researcher. The algorithms implemented in this library have been designed with usability and GPU efficiency in mind, and they can be added to any PyTorch model with just one extra line of code. We showcase the effectiveness of these tools on an application of optimisation on manifolds in the setting of time series analysis. In this setting, orthogonal and unitary optimisation is used to constraint and regularise recurrent models and avoid vanishing and exploding gradient problems. The algorithms designed for GeoTorch allow us to achieve state of the art results in the standard tests for this family of models. We use tools from comparison geometry to give bounds on quantities that are of interest in optimisation problems. In particular, we build on the work of (Kaul 1976) to give explicit bounds on the norm of the second derivative of the Riemannian exponential.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑