arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2002.04799cs.LGstat.ML
收藏

基于广义张量迹范数的深度多任务学习

Deep Multi-Task Learning via Generalized Tensor Trace Norm

Southern University of Science and Technology(南方科技大学);Nanjing University(南京大学)

Yi Zhang, Yu Zhang, Wei Wang

摘要:

迹范数因其能发现任务间模型参数的低秩结构而被广泛应用于多任务学习中。如今,随着大数据集的出现和深度学习技术的普及,张量迹范数已被用于深度多任务模型。然而,现有的张量迹范数不能发现所有的低秩结构,它们需要用户手动确定其组件的重要性。为了解决这两个问题,在本文中,我们提出了一个广义张量迹范数(GTTN)。GTTN被定义为所有可能的张量环的矩阵迹范数的凸组合,因此它可以发现所有可能的低秩结构。在诱导的目标函数中,我们将学习GTTN中的组合系数以自动确定重要性。在真实数据集上的实验证明了GTTN的有效性。

英文摘要:

The trace norm is widely used in multi-task learning as it can discover low-rank structures among tasks in terms of model parameters. Nowadays, with the emerging of big datasets and the popularity of deep learning techniques, tensor trace norms have been used for deep multi-task models. However, existing tensor trace norms cannot discover all the low-rank structures and they require users to manually determine the importance of their components. To solve those two issues together, in this paper, we propose a Generalized Tensor Trace Norm (GTTN). The GTTN is defined as a convex combination of matrix trace norms of all possible tensor flattenings and hence it can discover all the possible low-rank structures. In the induced objective function, we will learn combination coefficients in the GTTN to automatically determine the importance. Experiments on real-world datasets demonstrate the effectiveness of the proposed GTTN.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑