通过冗余信息学习不变图表示
Learning Invariant Graph Representations Through Redundant Information
AI总结:
本文提出RIG框架,通过冗余信息学习不变图表示,以实现分布外泛化。
摘要:
学习用于分布外(OOD)泛化的不变图表示仍然具有挑战性,因为学到的表示通常保留了虚假组件。为了解决这一挑战,本文介绍了一个来自信息论的新工具,称为部分信息分解(PID),它超越了经典信息论度量。我们指出现有方法在不变表示学习中仅依赖经典信息论度量的局限性,这促使我们精确关注通过PID获得的关于目标Y的冗余信息,该信息在虚假子图G_s和不变子图G_c之间共享。接下来,我们提出了一种新的多级优化框架,称为冗余引导不变图学习(RIG),它在最大化冗余信息的同时隔离虚假和因果子图,从而在多种分布偏移下实现OOD泛化。我们的方法依赖于在估计冗余信息的下界(这本身需要优化)和最大化它的同时最大化其他目标之间交替进行。在合成和现实世界图数据集上的实验展示了我们提出的RIG框架的泛化能力。
英文摘要:
Learning invariant graph representations for out-of-distribution (OOD) generalization remains challenging because the learned representations often retain spurious components. To address this challenge, this work introduces a new tool from information theory called Partial Information Decomposition (PID) that goes beyond classical information-theoretic measures. We identify limitations in existing approaches for invariant representation learning that solely rely on classical information-theoretic measures, motivating the need to precisely focus on redundant information about the target $Y$ shared between spurious subgraphs $G_s$ and invariant subgraphs $G_c$ obtained via PID. Next, we propose a new multi-level optimization framework that we call -- Redundancy-guided Invariant Graph learning (RIG) -- that maximizes redundant information while isolating spurious and causal subgraphs, enabling OOD generalization under diverse distribution shifts. Our approach relies on alternating between estimating a lower bound of redundant information (which itself requires an optimization) and maximizing it along with additional objectives. Experiments on both synthetic and real-world graph datasets demonstrate the generalization capabilities of our proposed RIG framework.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。