面向扩展网络的持续学习中拓扑感知嵌入记忆
Topology-aware Embedding Memory for Continual Learning on Expanding Networks
AI总结：
本文针对持续扩展网络中的记忆爆炸问题，提出参数解耦图神经网络与拓扑感知嵌入记忆框架，将记忆复杂度从 O(nd^L) 降至 O(n)，并利用伪训练效应设计覆盖最大化采样策略，显著提升类增量场景下的性能。
摘要：
基于记忆回放的技术在增量积累的欧几里得数据持续学习中取得了巨大成功。然而，直接将它们应用于持续扩展的网络会导致潜在的记忆爆炸问题，因为需要缓存代表性节点及其关联的拓扑邻域结构。为此，我们系统分析了记忆爆炸问题中的关键挑战，并提出了一个通用框架，即带有拓扑感知嵌入记忆（TEM）的参数解耦图神经网络（PDGNNs），以解决这一问题。所提出的框架不仅将记忆空间复杂度从 $\mathcal{O}(nd^L)$ 降低到 $\mathcal{O}(n)$（脚注：$n$：记忆预算，$d$：平均节点度，$L$：GNN 感受野半径），而且充分利用了拓扑信息进行记忆回放。具体来说，PDGNNs 通过拓扑感知嵌入（TEs）将可训练参数与计算自我子网络解耦，将自我子网络压缩为紧凑向量（即 TEs）以减少记忆消耗。基于这一框架，我们发现了扩展网络持续学习中独特的伪训练效应，这一效应促使我们开发了一种新颖的覆盖最大化采样策略，该策略可以在紧张的记忆预算下提升性能。充分的实证研究表明，通过解决记忆爆炸问题并将拓扑信息融入记忆回放，带有 TEM 的 PDGNNs 显著优于最先进的技术，尤其是在具有挑战性的类增量设置中。
英文摘要：
Memory replay based techniques have shown great success for continual learning with incrementally accumulated Euclidean data. Directly applying them to continually expanding networks, however, leads to the potential memory explosion problem due to the need to buffer representative nodes and their associated topological neighborhood structures. To this end, we systematically analyze the key challenges in the memory explosion problem, and present a general framework, \textit{i.e.}, Parameter Decoupled Graph Neural Networks (PDGNNs) with Topology-aware Embedding Memory (TEM), to tackle this issue. The proposed framework not only reduces the memory space complexity from $\mathcal{O}(nd^L)$ to $\mathcal{O}(n)$~\footnote{$n$: memory budget, $d$: average node degree, $L$: the radius of the GNN receptive field}, but also fully utilizes the topological information for memory replay. Specifically, PDGNNs decouple trainable parameters from the computation ego-subnetwork via \textit{Topology-aware Embeddings} (TEs), which compress ego-subnetworks into compact vectors (\textit{i.e.}, TEs) to reduce the memory consumption. Based on this framework, we discover a unique \textit{pseudo-training effect} in continual learning on expanding networks and this effect motivates us to develop a novel \textit{coverage maximization sampling} strategy that can enhance the performance with a tight memory budget. Thorough empirical studies demonstrate that, by tackling the memory explosion problem and incorporating topological information into memory replay, PDGNNs with TEM significantly outperform state-of-the-art techniques, especially in the challenging class-incremental setting.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
