arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 1206.4639cs.LGcs.AI
收藏

权矩阵的自适应正则化

Adaptive Regularization for Weight Matrices

Technion(以色列理工学院);Bar Ilan University(巴伊兰大学);Google research(谷歌研究院)

Koby Crammer, Gal Chechik

摘要:

用于学习权重向量上的分布的算法,例如AROW,最近被经验地证明在各种问题上具有最先进的性能,具有很强的理论可靠性。将这些算法扩展到矩阵模型带来了挑战,因为分布的协方差中的自由参数的数量随着矩阵的维度$n$而缩放为$n^4$,并且$n$在实际应用中往往很大。我们描述,分析和实验两个新的算法学习矩阵模型的分布。我们的第一个算法保持对角协方差的参数,可以处理大的协方差矩阵。第二个算法的因素的协方差捕获特征间的相关性,同时保持参数的数量在原始矩阵的大小线性。我们分析了这两种算法的错误界模型,并显示出优越的精度性能,我们的方法比其他算法在两个任务:检索相似的图像,并排名相似的文件。因子化算法具有更快的收敛速度。

英文摘要:

Algorithms for learning distributions over weight-vectors, such as AROW were recently shown empirically to achieve state-of-the-art performance at various problems, with strong theoretical guaranties. Extending these algorithms to matrix models pose challenges since the number of free parameters in the covariance of the distribution scales as $n^4$ with the dimension $n$ of the matrix, and $n$ tends to be large in real applications. We describe, analyze and experiment with two new algorithms for learning distribution of matrix models. Our first algorithm maintains a diagonal covariance over the parameters and can handle large covariance matrices. The second algorithm factors the covariance to capture inter-features correlation while keeping the number of parameters linear in the size of the original matrix. We analyze both algorithms in the mistake bound model and show a superior precision performance of our approach over other algorithms in two tasks: retrieving similar images, and ranking similar documents. The factored algorithm is shown to attain faster convergence rate.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑