arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2411.03025cs.LGcs.AI
收藏

DA-MoE:通过专家混合解决图级分析中的深度敏感性问题

DA-MoE: Addressing Depth-Sensitivity in Graph-Level Analysis through Mixture of Experts

Wuhan University(武汉大学);JD Explore Academy(京东探索研究院);Macquarie University(麦考瑞大学);Griffith University(格里菲斯大学)

Zelin Yao, Chuang Liu, Xianke Meng, Yibing Zhan, Jia Wu, Shirui Pan, Wenbin Hu

AI总结:

DA-MoE通过专家混合机制自适应调整GNN深度,解决图数据规模差异导致的深度敏感性问题,在TU和OGB基准上超越现有方法。

摘要:

图神经网络(GNN)在处理图结构数据方面日益受到欢迎。在现实场景中,同一数据集内的图数据在规模上可能存在显著差异。这种差异性导致了深度敏感性问题,即GNN层的最优深度取决于图数据的规模。经验上,对于较小的图,较少的层数足以进行消息传递,而较大的图通常需要更深的网络来捕获长距离依赖和全局特征。然而,现有方法通常对所有图使用固定数量的GNN层来生成表示,忽视了图结构数据中的深度敏感性问题。为了解决这一挑战,我们提出了深度自适应专家混合(DA-MoE)方法,该方法对GNN主干网络进行了两项主要改进:1)DA-MoE采用不同的GNN层,每一层被视为具有自身参数的专家。这种设计允许模型在不同尺度上灵活地聚合信息,有效解决了图数据中的深度敏感性问题。2)DA-MoE利用GNN来捕获结构信息,而不是门控网络中的线性投影。因此,门控网络使模型能够捕获数据中的复杂模式和依赖关系。通过利用这些改进,DA-MoE中的每个专家专门学习不同尺度下的不同图模式。此外,在TU数据集和开放图基准(OGB)上的综合实验表明,DA-MoE在各种任务上持续超越现有基线,包括图、节点和链接级别的分析。代码可在https://github.com/Celin-Yao/DA-MoE获取。

英文摘要:

Graph neural networks (GNNs) are gaining popularity for processing graph-structured data. In real-world scenarios, graph data within the same dataset can vary significantly in scale. This variability leads to depth-sensitivity, where the optimal depth of GNN layers depends on the scale of the graph data. Empirically, fewer layers are sufficient for message passing in smaller graphs, while larger graphs typically require deeper networks to capture long-range dependencies and global features. However, existing methods generally use a fixed number of GNN layers to generate representations for all graphs, overlooking the depth-sensitivity issue in graph structure data. To address this challenge, we propose the depth adaptive mixture of expert (DA-MoE) method, which incorporates two main improvements to GNN backbone: \textbf{1)} DA-MoE employs different GNN layers, each considered an expert with its own parameters. Such a design allows the model to flexibly aggregate information at different scales, effectively addressing the depth-sensitivity issue in graph data. \textbf{2)} DA-MoE utilizes GNN to capture the structural information instead of the linear projections in the gating network. Thus, the gating network enables the model to capture complex patterns and dependencies within the data. By leveraging these improvements, each expert in DA-MoE specifically learns distinct graph patterns at different scales. Furthermore, comprehensive experiments on the TU dataset and open graph benchmark (OGB) have shown that DA-MoE consistently surpasses existing baselines on various tasks, including graph, node, and link-level analyses. The code are available at \url{https://github.com/Celin-Yao/DA-MoE}.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑