EvoluNet:推进图上的动态非独立同分布(Non-IID)迁移学习
EvoluNet: Advancing Dynamic Non-IID Transfer Learning on Graphs
AI总结:
针对图上动态Non-IID迁移学习中域演化与动态差异的挑战,提出泛化界并构建EvoluNet框架,通过时序编码和动态域统一模块学习域不变表示,性能较最优模型最高提升12.1%。
摘要:
图上的非独立同分布(Non-IID)迁移学习在许多高风险领域至关重要。现有大多数工作假设源域和目标域的分布都是平稳的。然而,现实世界的图本质上是动态的,带来了域演化以及源域与目标域间动态差异的挑战。为弥补这一研究空白,我们将问题转向动态场景,并提出问题:给定前T个时间步观测到的标签丰富的源图和标签稀缺的目标图,我们如何有效刻画不断演化的域差异,并优化即将到来的T+1时间步目标域的泛化性能?为解答该问题,我们提出了图上动态Non-IID迁移学习的泛化界,该泛化界表明泛化性能由源图与目标图之间的域演化和域差异主导。受理论结果启发,我们提出了名为EvoluNet的新型通用框架。它利用基于transformer的时序编码模块对演化域的时序信息建模,随后使用动态域统一模块高效学习源域和目标域间的域不变表示。最终,EvoluNet的性能比现有最优模型最高提升12.1%,证明了其在从动态源图向动态目标图迁移知识方面的有效性。
英文摘要:
Non-IID transfer learning on graphs is crucial in many high-stakes domains. The majority of existing works assume stationary distribution for both source and target domains. However, real-world graphs are intrinsically dynamic, presenting challenges in terms of domain evolution and dynamic discrepancy between source and target domains. To bridge the gap, we shift the problem to the dynamic setting and pose the question: given the label-rich source graphs and the label-scarce target graphs both observed in previous T timestamps, how can we effectively characterize the evolving domain discrepancy and optimize the generalization performance of the target domain at the incoming T+1 timestamp? To answer it, we propose a generalization bound for dynamic non-IID transfer learning on graphs, which implies the generalization performance is dominated by domain evolution and domain discrepancy between source and target graphs. Inspired by the theoretical results, we introduce a novel generic framework named EvoluNet. It leverages a transformer-based temporal encoding module to model temporal information of the evolving domains and then uses a dynamic domain unification module to efficiently learn domain-invariant representations across the source and target domains. Finally, EvoluNet outperforms the state-of-the-art models by up to 12.1%, demonstrating its effectiveness in transferring knowledge from dynamic source graphs to dynamic target graphs.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。