arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2602.18591cs.LG
收藏

任务亲和性集成预测以实现高效多任务学习

Ensemble Prediction of Task Affinity for Efficient Multi-Task Learning

Pennsylvania State University(宾夕法尼亚州立大学);College of William & Mary(威廉与玛丽学院)

Afiya Ayman, Ayan Mukhopadhyay, Aron Laszka

AI总结:

ETAP通过集成任务亲和性预测器,提升多任务学习的性能预测和任务分组效率。

摘要:

多任务学习(MTL)中的一个基本问题是确定应一起学习的任务组。由于为所有可能的任务组合训练MTL模型对于大规模任务集来说成本过高,高效且有效的任务分组的关键组成部分是预测一组任务是否能从一起学习中获益,这通过每任务性能提升来衡量。在本文中,我们提出了ETAP(集成任务亲和性预测器),一个可扩展的框架,该框架整合了原理性与数据驱动的估计器来预测MTL性能提升。首先,我们考虑MTL模型中共享参数的梯度更新来测量一对任务之间的亲和性,作为基于这些任务的参数更新的相似性。这种线性估计器,我们称之为亲和分数,自然扩展到估计任务组内的亲和性。其次,为了改进这些估计,我们训练预测器应用非线性变换并纠正残差误差,捕捉复杂和非线性的任务关系。我们在这有限数量的任务组上训练这些预测器,通过多任务学习为每个组获取真实增益值。我们在基准数据集上展示了ETAP能够提高MTL增益预测并实现更有效的任务分组,优于现有最先进基线方法,在各种应用领域均表现优异。

英文摘要:

A fundamental problem in multi-task learning (MTL) is identifying groups of tasks that should be learned together. Since training MTL models for all possible combinations of tasks is prohibitively expensive for large task sets, a crucial component of efficient and effective task grouping is predicting whether a group of tasks would benefit from learning together, measured as per-task performance gain over single-task learning. In this paper, we propose ETAP (Ensemble Task Affinity Predictor), a scalable framework that integrates principled and data-driven estimators to predict MTL performance gains. First, we consider the gradient-based updates of shared parameters in an MTL model to measure the affinity between a pair of tasks as the similarity between the parameter updates based on these tasks. This linear estimator, which we call affinity score, naturally extends to estimating affinity within a group of tasks. Second, to refine these estimates, we train predictors that apply non-linear transformations and correct residual errors, capturing complex and non-linear task relationships. We train these predictors on a limited number of task groups for which we obtain ground-truth gain values via multi-task learning for each group. We demonstrate on benchmark datasets that ETAP improves MTL gain prediction and enables more effective task grouping, outperforming state-of-the-art baselines across diverse application domains.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑