arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2409.16645cs.LGcs.AI
收藏

基于几何对齐的多任务学习中的任务添加

Task Addition in Multi-Task Learning by Geometrical Alignment

LG AI Research(LG AI研究院)

Soorin Yim, Dae-Woong Jeong, Sung Moon Ko, Sumin Lee, Hyunseung Kim, Chanhui Lee, Sehui Han

AI总结:

针对分子性质预测中GATE算法扩展到多任务时计算成本高的限制,提出一种任务添加方法,通过大规模数据集预训练后添加任务特定模块,在有限数据下以相当计算成本显著提升性能。

摘要:

在有限数据上训练深度学习模型同时保持泛化能力,是分子性质预测的基本挑战之一。一种有效的解决方案是将从丰富数据集中提取的知识迁移到数据稀缺的数据集上。最近,一种名为几何对齐迁移编码器(GATE)的新算法被提出,它通过对齐任务特定潜在空间的几何形状来实现软参数共享。然而,由于计算成本,GATE在扩展到多任务时面临限制。在本研究中,我们提出了一种针对GATE的任务添加方法,以在最小化计算复杂度的同时提高有限数据下目标任务的表现。该方法通过在大型数据集上进行有监督多任务预训练,随后为每个目标任务添加并训练任务特定模块来实现。实验表明,GATE的任务添加策略在计算成本相当的情况下,性能优于常规多任务方法。

英文摘要:

Training deep learning models on limited data while maintaining generalization is one of the fundamental challenges in molecular property prediction. One effective solution is transferring knowledge extracted from abundant datasets to those with scarce data. Recently, a novel algorithm called Geometrically Aligned Transfer Encoder (GATE) has been introduced, which uses soft parameter sharing by aligning the geometrical shapes of task-specific latent spaces. However, GATE faces limitations in scaling to multiple tasks due to computational costs. In this study, we propose a task addition approach for GATE to improve performance on target tasks with limited data while minimizing computational complexity. It is achieved through supervised multi-task pre-training on a large dataset, followed by the addition and training of task-specific modules for each target task. Our experiments demonstrate the superior performance of the task addition strategy for GATE over conventional multi-task methods, with comparable computational costs.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑