arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2602.09258cs.LG
收藏

泛化GNN:基于令牌化的专家混合

Generalizing GNNs with Tokenized Mixture of Experts

University of Connecticut Storrs(康涅狄格大学斯特劳斯分校);University of Notre Dame(Notre Dame 大学);University of Virginia(弗吉尼亚大学);Northwestern University Evanston(北western 大学埃文斯顿分校)

Xiaoguang Guo, Zehong Wang, Jiazheng Li, Shawn Spitzel, Qi Yang, Kaize Ding, Jundong Li, Chuxu Zhang

AI总结:

针对图神经网络部署时稳定性与泛化性的权衡,提出STEM-GNN框架,通过令牌化专家混合编码器、向量量化接口和Lipschitz正则化头实现三方面平衡,在多种分布偏移和扰动下提升鲁棒性。

摘要:

部署的图神经网络(GNN)在部署时是冻结的,但必须适应干净数据,在分布偏移下泛化,并对扰动保持稳定。我们表明静态推理引入了一个基本权衡:提高稳定性需要减少对偏移敏感特征的依赖,留下一个不可约的最坏情况泛化下限。实例条件路由可以打破这个上限,但很脆弱,因为偏移可能误导路由,扰动可能使路由波动。我们通过两个分解来捕捉这些效应:覆盖与选择分离,以及基础敏感性与波动放大分离。基于这些见解,我们提出了STEM-GNN,一个预训练-微调框架,包含一个用于多样化计算路径的专家混合编码器,一个用于稳定编码器到头部信号的向量量化令牌接口,以及一个用于限制输出放大的Lipschitz正则化头部。在九个节点、链接和图基准测试中,STEM-GNN实现了更强的三方面平衡,提高了对度/同质性偏移以及特征/边损坏的鲁棒性,同时在干净图上保持竞争力。

英文摘要:

Deployed graph neural networks (GNNs) are frozen at deployment yet must fit clean data, generalize under distribution shifts, and remain stable to perturbations. We show that static inference induces a fundamental tradeoff: improving stability requires reducing reliance on shift-sensitive features, leaving an irreducible worst-case generalization floor. Instance-conditional routing can break this ceiling, but is fragile because shifts can mislead routing and perturbations can make routing fluctuate. We capture these effects via two decompositions separating coverage vs selection, and base sensitivity vs fluctuation amplification. Based on these insights, we propose STEM-GNN, a pretrain-then-finetune framework with a mixture-of-experts encoder for diverse computation paths, a vector-quantized token interface to stabilize encoder-to-head signals, and a Lipschitz-regularized head to bound output amplification. Across nine node, link, and graph benchmarks, STEM-GNN achieves a stronger three-way balance, improving robustness to degree/homophily shifts and to feature/edge corruptions while remaining competitive on clean graphs.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑