arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2004.14034cs.LGcs.AIstat.ML
收藏

多任务回归问题的新兴关系网络与任务嵌入

Emerging Relation Network and Task Embedding for Multi-Task Regression Problems

University of Kassel(卡塞尔大学)

Jens Schreiber, Bernhard Sick

摘要:

多任务学习(mtl)在计算机视觉和自然语言处理的许多应用中提供了最先进的结果。与单任务学习(stl)不同,mtl允许利用相关任务之间的知识来改善主任务(与辅助任务相比)或所有任务的预测结果。然而,考虑到mtl的最新进展,将mtl架构应用于回归和时间序列问题的比较研究数量有限。一个有趣的非线性问题是预测可再生能源发电厂的预期发电量。因此,本文对以下最新且重要的mtl架构进行了比较研究:硬参数共享、十字绣网络、水闸网络(SN)。他们比较了一个多层感知器模型的相似大小的stl设置。此外,我们提供了一个简单的,但有效的方法来建模任务特定的信息,通过嵌入层在多层感知器,称为任务嵌入。此外,我们还介绍了一种新的多目标语言体系结构新兴关系网络(ern),它可以被认为是水闸网络的扩展。对于太阳能数据集,任务嵌入实现了14.9%的最佳平均改善。太阳数据集上的ern和sn的平均改善幅度相似,分别为14.7%和14.8%。在风力发电数据集上,只有ern实现了高达7.7%的显着改善。结果表明,ern是有益的,当任务只是松散相关的预测问题是更加非线性。相反,所提出的任务嵌入是有利的,当任务是强相关的。此外,任务嵌入提供了一种有效的方法,与其他mtl架构相比,减少了计算工作量。

英文摘要:

Multi-task learning (mtl) provides state-of-the-art results in many applications of computer vision and natural language processing. In contrast to single-task learning (stl), mtl allows for leveraging knowledge between related tasks improving prediction results on the main task (in contrast to an auxiliary task) or all tasks. However, there is a limited number of comparative studies on applying mtl architectures for regression and time series problems taking recent advances of mtl into account. An interesting, non-linear problem is the forecast of the expected power generation for renewable power plants. Therefore, this article provides a comparative study of the following recent and important mtl architectures: Hard parameter sharing, cross-stitch network, sluice network (sn). They are compared to a multi-layer perceptron model of similar size in an stl setting. Additionally, we provide a simple, yet effective approach to model task specific information through an embedding layer in an multi-layer perceptron, referred to as task embedding. Further, we introduce a new mtl architecture named emerging relation network (ern), which can be considered as an extension of the sluice network. For a solar power dataset, the task embedding achieves the best mean improvement with 14.9%. The mean improvement of the ern and the sn on the solar dataset is of similar magnitude with 14.7% and 14.8%. On a wind power dataset, only the ern achieves a significant improvement of up to 7.7%. Results suggest that the ern is beneficial when tasks are only loosely related and the prediction problem is more non-linear. Contrary, the proposed task embedding is advantageous when tasks are strongly correlated. Further, the task embedding provides an effective approach with reduced computational effort compared to other mtl architectures.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑