STG-MTL:基于数据图谱的多任务学习可扩展任务分组
STG-MTL: Scalable Task Grouping for Multi-Task Learning Using Data Map
AI总结:
针对多任务学习任务分组指数级搜索导致的可扩展性难题,提出基于数据图谱的数据驱动分组方法,理论证明可扩展性更优,实验在CIFAR100上支持多达100个任务,性能更好且分组合理。
摘要:
多任务学习(MTL)是一种强大的技术,因其相比传统单任务学习(STL)的性能提升而广受欢迎。然而,MTL 常常具有挑战性,因为可能存在的任务分组数量呈指数级增长,这使得选择最佳分组变得困难,因为某些分组可能由于任务间的负干扰而导致性能下降。这就是现有解决方案严重遭受可扩展性问题困扰的原因,限制了任何实际应用。在本文中,我们提出了一种新的数据驱动方法,以应对这些挑战,并提供一种基于重新提出的数据驱动特征——数据图谱(Data Maps)的、可扩展且模块化的分类任务分组解决方案,该特征在 MTL 训练过程中捕获每个分类任务的训练动态。通过与其他技术的理论比较,我们成功证明了我们的方法具有优越的可扩展性。我们的实验展示了更好的性能,并验证了该方法的有效性,即使是在前所未有的任务数量(CIFAR100 上多达 100 个任务)下也是如此。作为首个处理如此数量任务的研究,我们在所得分组上的比较显示,其分组与数据集 CIFAR100 中提到的分组相似。最后,我们提供了一个模块化实现,以便于集成和测试,并附带了来自多个数据集和任务的示例。
英文摘要:
Multi-Task Learning (MTL) is a powerful technique that has gained popularity due to its performance improvement over traditional Single-Task Learning (STL). However, MTL is often challenging because there is an exponential number of possible task groupings, which can make it difficult to choose the best one because some groupings might produce performance degradation due to negative interference between tasks. That is why existing solutions are severely suffering from scalability issues, limiting any practical application. In our paper, we propose a new data-driven method that addresses these challenges and provides a scalable and modular solution for classification task grouping based on a re-proposed data-driven features, Data Maps, which capture the training dynamics for each classification task during the MTL training. Through a theoretical comparison with other techniques, we manage to show that our approach has the superior scalability. Our experiments show a better performance and verify the method's effectiveness, even on an unprecedented number of tasks (up to 100 tasks on CIFAR100). Being the first to work on such number of tasks, our comparisons on the resulting grouping shows similar grouping to the mentioned in the dataset, CIFAR100. Finally, we provide a modular implementation for easier integration and testing, with examples from multiple datasets and tasks.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。