arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2103.02546cs.LGcs.AI
收藏

利用语义信息的多任务学习

Multi-task Learning by Leveraging the Semantic Information

University of Western Ontario(西安大略大学);Vector Institute(向量研究所)

Fan Zhou, Brahim Chaib-draa, Boyu Wang

摘要:

多任务学习的一个关键目标是调整任务之间的分布,以便它们之间的信息可以传输和共享。然而,现有的方法只关注边缘特征分布的匹配,而忽略了语义信息,这可能会影响学习性能。为了解决这个问题,我们提出了利用标签信息在多任务学习中探索任务之间的语义条件关系。首先,基于Jensen-Shannon发散度的概念,从理论上分析了多任务学习的泛化界,为多任务学习中标签信息的价值提供了新的见解.我们的分析还导致了一个具体的算法,共同匹配的语义分布和控制标签分布的分歧。为了证实所提出的方法的有效性,我们首先比较了算法与几个基线在一些基准,然后测试的算法在标签空间移位的条件下。实验结果表明,所提出的方法可以优于大多数基线,并达到最先进的性能,特别是显示的好处下的标签偏移的条件。

英文摘要:

One crucial objective of multi-task learning is to align distributions across tasks so that the information between them can be transferred and shared. However, existing approaches only focused on matching the marginal feature distribution while ignoring the semantic information, which may hinder the learning performance. To address this issue, we propose to leverage the label information in multi-task learning by exploring the semantic conditional relations among tasks. We first theoretically analyze the generalization bound of multi-task learning based on the notion of Jensen-Shannon divergence, which provides new insights into the value of label information in multi-task learning. Our analysis also leads to a concrete algorithm that jointly matches the semantic distribution and controls label distribution divergence. To confirm the effectiveness of the proposed method, we first compare the algorithm with several baselines on some benchmarks and then test the algorithms under label space shift conditions. Empirical results demonstrate that the proposed method could outperform most baselines and achieve state-of-the-art performance, particularly showing the benefits under the label shift conditions.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑