arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2004.13930cs.LGstat.ML
收藏

任务-特征协同学习及其在个性化属性预测中的应用

Task-Feature Collaborative Learning with Application to Personalized Attribute Prediction

Institute of Information Engineering, Chinese Academy of Sciences(中国科学院信息工程研究所);University of Chinese Academy of Sciences(中国科学院大学);Institute of Computing Technology, Chinese Academy of Sciences(中国科学院计算技术研究所);Peng Cheng Laboratory(鹏城实验室)

Zhiyong Yang, Qianqian Xu, Xiaochun Cao, Qingming Huang

摘要:

多任务学习作为一种有效的学习范式,在训练样本不足的情况下,鼓励多个相关任务之间的知识共享,以提高整体性能。在MTL中,一个主要的挑战来自于这样的现象,即与不同的和困难的任务共享知识,称为负迁移,往往会导致更差的性能。虽然已有大量的研究针对负迁移进行了研究,但大多数现有的方法仅将迁移关系建模为任务相关性,而没有考虑跨特征和任务的迁移。与现有的方法不同,我们的目标是在功能和任务之间协同减轻负迁移。为此,我们提出了一种新的多任务学习方法,称为任务特征协作学习(TFCL)。具体来说,我们首先提出了一个基础模型与异构块对角结构正则化,利用协作分组的功能和任务,并抑制组间知识共享。然后,我们提出了一种优化方法的模型。大量的理论分析表明,我们提出的方法具有以下优点:(a)它具有全局收敛性和(b)它提供了一个块对角结构恢复保证。作为一个实际的扩展,我们通过允许重叠的功能和区分硬任务来扩展基础模型。我们进一步将其应用到个性化的属性预测问题与细粒度的用户行为建模。最后,在模拟数据集和真实数据集上的实验结果表明了该方法的有效性

英文摘要:

As an effective learning paradigm against insufficient training samples, Multi-Task Learning (MTL) encourages knowledge sharing across multiple related tasks so as to improve the overall performance. In MTL, a major challenge springs from the phenomenon that sharing the knowledge with dissimilar and hard tasks, known as negative transfer, often results in a worsened performance. Though a substantial amount of studies have been carried out against the negative transfer, most of the existing methods only model the transfer relationship as task correlations, with the transfer across features and tasks left unconsidered. Different from the existing methods, our goal is to alleviate negative transfer collaboratively across features and tasks. To this end, we propose a novel multi-task learning method called Task-Feature Collaborative Learning (TFCL). Specifically, we first propose a base model with a heterogeneous block-diagonal structure regularizer to leverage the collaborative grouping of features and tasks and suppressing inter-group knowledge sharing. We then propose an optimization method for the model. Extensive theoretical analysis shows that our proposed method has the following benefits: (a) it enjoys the global convergence property and (b) it provides a block-diagonal structure recovery guarantee. As a practical extension, we extend the base model by allowing overlapping features and differentiating the hard tasks. We further apply it to the personalized attribute prediction problem with fine-grained modeling of user behaviors. Finally, experimental results on both simulated dataset and real-world datasets demonstrate the effectiveness of our proposed method

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑