arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 1404.7195cs.LG
收藏

旋转矩阵和Hessian矩阵的快速逼近

Fast Approximation of Rotations and Hessians matrices

Courant Institute of Mathematical Sciences(库朗数学科学研究所);New York University(纽约大学)

Michael Mathieu, Yann LeCun

摘要:

介绍了一种新的旋转矩阵的表示和近似方法。该方法表示具有线性复杂度的旋转矩阵$Q$的近似,即具有以类似FFT的方式布置的坐标对上的$\frac{1}{2}n\lg(n)$旋转。使用梯度下降“学习”近似。它允许将对称矩阵$H$表示为$QDQ^T$,其中$D$是对角矩阵。它可以用来近似高斯模型的协方差矩阵,以加快推理,或通过将参数的变化与优化过程之后的轨迹上的梯度变化相关联来估计和跟踪目标函数的逆Hessian。实验进行了近似合成矩阵,真实数据的协方差矩阵,和Hessian矩阵的目标函数涉及机器学习问题。

英文摘要:

A new method to represent and approximate rotation matrices is introduced. The method represents approximations of a rotation matrix $Q$ with linearithmic complexity, i.e. with $\frac{1}{2}n\lg(n)$ rotations over pairs of coordinates, arranged in an FFT-like fashion. The approximation is "learned" using gradient descent. It allows to represent symmetric matrices $H$ as $QDQ^T$ where $D$ is a diagonal matrix. It can be used to approximate covariance matrix of Gaussian models in order to speed up inference, or to estimate and track the inverse Hessian of an objective function by relating changes in parameters to changes in gradient along the trajectory followed by the optimization procedure. Experiments were conducted to approximate synthetic matrices, covariance matrices of real data, and Hessian matrices of objective functions involved in machine learning problems.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑