学习为分子任务分组辅助数据集
Learning to Group Auxiliary Datasets for Molecule
AI总结:
针对小分子数据集标注不足、辅助数据集联合训练易出现负迁移的问题,提出MolGroup方法,通过拆分任务与结构亲和性、结合双层优化路由机制预测最优辅助数据集组合,显著提升分子模型性能。
摘要:
小分子数据集的标注可用性有限,给机器学习模型带来了挑战。为解决这一问题,一种常见策略是结合额外的辅助数据集开展训练。但数据量更多并不总能保证性能提升,当目标数据集与辅助分子数据集的知识存在差异或相互矛盾时,就可能发生负迁移。鉴于此,找出与目标数据集联合训练时能带来增益的辅助分子数据集,仍是一个关键且未解决的问题。\n通过实证分析我们发现,结合图结构相似性与任务相似性,可以作为识别高亲和性辅助数据集的更可靠指标。受这一发现启发,我们提出MolGroup,该方法将数据集亲和性拆分为任务亲和性与结构亲和性,以此预测每个辅助分子数据集的潜在增益。MolGroup通过一个经双层优化框架优化的路由机制实现这一目标:在元梯度的支撑下,路由机制以最大化目标数据集性能为方向被优化,并将亲和性量化为门控分数。由此,MolGroup能够为每个目标数据集预测最优的辅助数据集组合。\n我们的大量实验证明了MolGroup的效率与有效性:在11个目标分子数据集上,使用MolGroup选出的分子数据集组训练的GIN/Graphormer模型,性能平均提升了4.41%与3.47%。
英文摘要:
The limited availability of annotations in small molecule datasets presents a challenge to machine learning models. To address this, one common strategy is to collaborate with additional auxiliary datasets. However, having more data does not always guarantee improvements. Negative transfer can occur when the knowledge in the target dataset differs or contradicts that of the auxiliary molecule datasets. In light of this, identifying the auxiliary molecule datasets that can benefit the target dataset when jointly trained remains a critical and unresolved problem. Through an empirical analysis, we observe that combining graph structure similarity and task similarity can serve as a more reliable indicator for identifying high-affinity auxiliary datasets. Motivated by this insight, we propose MolGroup, which separates the dataset affinity into task and structure affinity to predict the potential benefits of each auxiliary molecule dataset. MolGroup achieves this by utilizing a routing mechanism optimized through a bi-level optimization framework. Empowered by the meta gradient, the routing mechanism is optimized toward maximizing the target dataset's performance and quantifies the affinity as the gating score. As a result, MolGroup is capable of predicting the optimal combination of auxiliary datasets for each target dataset. Our extensive experiments demonstrate the efficiency and effectiveness of MolGroup, showing an average improvement of 4.41%/3.47% for GIN/Graphormer trained with the group of molecule datasets selected by MolGroup on 11 target molecule datasets.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。