arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2008.05437cs.LGstat.ML
收藏

张量网络结构的自适应学习

Adaptive Learning of Tensor Network Structures

Mila(米拉人工智能研究所);DIRO(计算机科学与运筹学系);Université de Montréal(蒙特利尔大学);CCAI chair(加拿大人工智能讲席)

Meraj Hashemizadeh, Michelle Liu, Jacob Miller, Guillaume Rabusseau

摘要:

张量网络(TN)提供了一个强大的框架来有效地表示非常高维的对象。TN最近显示了它们在机器学习应用中的潜力,并提供了常见张量分解模型(如Tucker,张量训练(TT)和张量环(TR))的统一视图。然而,从给定任务的数据中识别最佳张量网络结构具有挑战性。在这项工作中,我们利用TN形式主义开发一个通用的和有效的自适应算法,共同学习的结构和参数的TN数据。我们的方法是基于一个简单的贪婪的方法,从一个秩张量开始,并连续识别最有前途的张量网络边缘的小秩增量。我们的算法可以自适应地识别TN结构与少量的参数,有效地优化任何可微的目标函数。在张量分解、张量补全和模型压缩任务上的实验证明了该算法的有效性。特别是,我们的方法优于最先进的进化拓扑搜索[Li和Sun,2020],用于图像的张量分解(同时快几个数量级),并找到有效的张量网络结构来压缩神经网络,优于流行的基于TT的方法[Novikov等人,2015年]。

英文摘要:

Tensor Networks (TN) offer a powerful framework to efficiently represent very high-dimensional objects. TN have recently shown their potential for machine learning applications and offer a unifying view of common tensor decomposition models such as Tucker, tensor train (TT) and tensor ring (TR). However, identifying the best tensor network structure from data for a given task is challenging. In this work, we leverage the TN formalism to develop a generic and efficient adaptive algorithm to jointly learn the structure and the parameters of a TN from data. Our method is based on a simple greedy approach starting from a rank one tensor and successively identifying the most promising tensor network edges for small rank increments. Our algorithm can adaptively identify TN structures with small number of parameters that effectively optimize any differentiable objective function. Experiments on tensor decomposition, tensor completion and model compression tasks demonstrate the effectiveness of the proposed algorithm. In particular, our method outperforms the state-of-the-art evolutionary topology search [Li and Sun, 2020] for tensor decomposition of images (while being orders of magnitude faster) and finds efficient tensor network structures to compress neural networks outperforming popular TT based approaches [Novikov et al., 2015].

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑