arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 1708.07918cs.LGcs.AIcs.CLstat.ML
收藏

用于深度多任务学习的鲁棒任务聚类

Robust Task Clustering for Deep Many-Task Learning

IBM Research(IBM研究院);IBM T. J. Watson Research Center(IBM托马斯·J·沃森研究中心)

Mo Yu, Xiaoxiao Guo, Jinfeng Yi, Shiyu Chang, Saloni Potdar, Gerald Tesauro, Haoyu Wang, Bowen Zhou

摘要:

我们研究了多任务环境下基于深度学习的多任务和少样本学习的任务聚类。针对深度学习场景,提出了一种新的任务相似性度量方法--跨任务迁移性能矩阵。虽然这个矩阵为我们提供了关键的信息,任务之间的相似性,其不对称的属性和不可靠的性能分数会影响传统的聚类方法不利。此外,不确定的任务对,即,具有极不对称的转移分数的那些可能共同地误导聚类算法以输出不准确的任务划分。为了克服这些局限性,我们提出了一种新的任务聚类算法,通过使用矩阵完成技术。该算法根据任务对聚类成员的确定性,构造了一个部分可观察的相似性矩阵。然后,我们使用矩阵完成算法来完成相似性矩阵。我们的理论分析表明,在温和的约束条件下,该算法将完美地恢复潜在的“真正的”相似性矩阵的概率很高。我们的研究结果表明,新的任务聚类方法可以发现任务集群,用于训练灵活和优越的神经网络模型,在多任务学习设置的情感分类和对话意图分类任务。我们的任务聚类方法还扩展了基于度量的少样本学习方法,以适应多个度量,这表明了经验优势,当任务是不同的。

英文摘要:

We investigate task clustering for deep-learning based multi-task and few-shot learning in a many-task setting. We propose a new method to measure task similarities with cross-task transfer performance matrix for the deep learning scenario. Although this matrix provides us critical information regarding similarity between tasks, its asymmetric property and unreliable performance scores can affect conventional clustering methods adversely. Additionally, the uncertain task-pairs, i.e., the ones with extremely asymmetric transfer scores, may collectively mislead clustering algorithms to output an inaccurate task-partition. To overcome these limitations, we propose a novel task-clustering algorithm by using the matrix completion technique. The proposed algorithm constructs a partially-observed similarity matrix based on the certainty of cluster membership of the task-pairs. We then use a matrix completion algorithm to complete the similarity matrix. Our theoretical analysis shows that under mild constraints, the proposed algorithm will perfectly recover the underlying "true" similarity matrix with a high probability. Our results show that the new task clustering method can discover task clusters for training flexible and superior neural network models in a multi-task learning setup for sentiment classification and dialog intent classification tasks. Our task clustering approach also extends metric-based few-shot learning methods to adapt multiple metrics, which demonstrates empirical advantages when the tasks are diverse.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑