arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2111.15422cs.LG
收藏

连续图表示学习的层次原型网络

Hierarchical Prototype Networks for Continual Graph Representation Learning

The University of Sydney(悉尼大学);University of Connecticut(康涅狄格大学)

Xikun Zhang, Dongjin Song, Dacheng Tao

摘要:

尽管在图表示学习中取得了显著的进步,但是很少关注更实际的持续学习场景,其中新的节点类别(例如,引文网络中的新研究领域、或共同购买网络中的新产品类型)及其相关联的边缘不断出现,导致对先前类别的灾难性遗忘。现有的方法要么忽略了丰富的拓扑信息,要么为了稳定性而牺牲了塑性。为此,我们提出了分层原型网络(HPN),它以原型的形式提取不同层次的抽象知识来表示不断扩展的图形。具体来说,我们首先利用一组原子特征提取器(AFE)来编码目标节点的元素属性信息和拓扑结构。接下来,我们开发HPN自适应地选择相关的AFE,并表示每个节点与三个层次的原型。通过这种方式,每当给定一个新的节点类别时,只有每个级别的相关AFE和原型将被激活和优化,而其他AFE和原型则保持不间断,以保持现有节点的性能。从理论上讲,我们首先证明了HPN的内存消耗是有界的,无论遇到多少任务。然后,我们证明了在温和的约束下,学习新的任务不会改变与以前的数据匹配的原型,从而消除了遗忘问题。在五个数据集上的实验支持了理论结果,表明HPN不仅优于最先进的基线技术,而且消耗的内存相对较少。

英文摘要:

Despite significant advances in graph representation learning, little attention has been paid to the more practical continual learning scenario in which new categories of nodes (e.g., new research areas in citation networks, or new types of products in co-purchasing networks) and their associated edges are continuously emerging, causing catastrophic forgetting on previous categories. Existing methods either ignore the rich topological information or sacrifice plasticity for stability. To this end, we present Hierarchical Prototype Networks (HPNs) which extract different levels of abstract knowledge in the form of prototypes to represent the continuously expanded graphs. Specifically, we first leverage a set of Atomic Feature Extractors (AFEs) to encode both the elemental attribute information and the topological structure of the target node. Next, we develop HPNs to adaptively select relevant AFEs and represent each node with three levels of prototypes. In this way, whenever a new category of nodes is given, only the relevant AFEs and prototypes at each level will be activated and refined, while others remain uninterrupted to maintain the performance over existing nodes. Theoretically, we first demonstrate that the memory consumption of HPNs is bounded regardless of how many tasks are encountered. Then, we prove that under mild constraints, learning new tasks will not alter the prototypes matched to previous data, thereby eliminating the forgetting problem. The theoretical results are supported by experiments on five datasets, showing that HPNs not only outperform state-of-the-art baseline techniques but also consume relatively less memory.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑