arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2406.04963cs.LGcs.AI
收藏

学习散度场以实现移位鲁棒的图表示

Learning Divergence Fields for Shift-Robust Graph Representations

MoE Lab of AI, Shanghai Jiao Tong University(上海交通大学教育部人工智能重点实验室)

Qitian Wu, Fan Nie, Chenxiao Yang, Junchi Yan

AI总结:

本文提出一种带可学习散度场的几何扩散模型,通过随机扩散率和因果推断学习跨领域稳定的相互依赖模式,并给出GCN、GAT和Transformer的广义版本,以提升图模型在分布偏移下的泛化能力。

摘要:

现实世界的数据生成通常涉及某些几何结构(例如图),这些结构会引发实例级的相互依赖。这一特性使得学习模型的泛化变得更加困难,因为复杂的相互依赖模式会影响数据生成分布,并且可能从训练阶段到测试阶段发生变化。在这项工作中,我们针对具有相互依赖数据的挑战性泛化问题,提出了一种带有可学习散度场的几何扩散模型。我们将扩散方程推广为在每个时间步具有随机扩散率的形式,旨在捕捉相互依赖数据之间多方面的信息流。此外,我们通过因果推断推导出一个新的学习目标,该目标可以引导模型学习跨领域不敏感的、可泛化的相互依赖模式。在具体实现方面,我们引入了三种模型实例,它们可以分别被视为 GCN、GAT 和 Transformers 的广义版本,并具有对分布偏移的高级鲁棒性。我们在多样的真实世界数据集上展示了它们在分布外泛化方面的良好效果。

英文摘要:

Real-world data generation often involves certain geometries (e.g., graphs) that induce instance-level interdependence. This characteristic makes the generalization of learning models more difficult due to the intricate interdependent patterns that impact data-generative distributions and can vary from training to testing. In this work, we propose a geometric diffusion model with learnable divergence fields for the challenging generalization problem with interdependent data. We generalize the diffusion equation with stochastic diffusivity at each time step, which aims to capture the multi-faceted information flows among interdependent data. Furthermore, we derive a new learning objective through causal inference, which can guide the model to learn generalizable patterns of interdependence that are insensitive across domains. Regarding practical implementation, we introduce three model instantiations that can be considered as the generalized versions of GCN, GAT, and Transformers, respectively, which possess advanced robustness against distribution shifts. We demonstrate their promising efficacy for out-of-distribution generalization on diverse real-world datasets.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑