arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2103.00164cs.LGcs.CG
收藏

FeatureNorm:面向动态图嵌入的L2特征规范化

FeatureNorm: L2 Feature Normalization for Dynamic Graph Embedding

The Chinese University of Hong Kong(香港中文大学)

Menglin Yang, Ziqiao Meng, Irwin King

摘要:

动态图出现在大量实际场景中,例如社交网络、通信网络和金融交易网络。给定一个动态图,学习一个图表示是基本的和必不可少的,该图表示不仅要保持结构上的接近性,而且还要共同捕获时间演变的模式。近年来,图卷积网络在非欧应用领域得到了广泛的研究和应用。GCN的主要成功之处在于它近似于拉普拉斯平滑,特别是在处理依赖关系和在节点内传递消息方面。事实上,这种平滑技术不仅可以鼓励必须链接的节点对变得更接近,而且还可以推动不能链接的节点对收缩在一起,这可能会导致严重的特征收缩或过度平滑问题,特别是当在多个层或步骤中堆叠图卷积时。对于学习时间演化模式,一个自然的解决方案是保留历史状态并将其与当前交互相结合以获得最新的表示。那么,按照目前流行的方法显式或隐式地叠加图卷积时,会出现严重的特征收缩或过平滑问题,使得节点过于相似而无法区分。为了解决动态图嵌入中的这个问题,首先分析了节点嵌入空间的收缩特性,然后设计了一种简单而通用的方法,该方法利用L2特征归一化约束将所有节点重新缩放到单位球的超球面中,使得节点不会收缩在一起,而相似的节点仍然可以靠近。在四个真实动态图数据集上的实验结果表明了该方法的有效性。

英文摘要:

Dynamic graphs arise in a plethora of practical scenarios such as social networks, communication networks, and financial transaction networks. Given a dynamic graph, it is fundamental and essential to learn a graph representation that is expected not only to preserve structural proximity but also jointly capture the time-evolving patterns. Recently, graph convolutional network (GCN) has been widely explored and used in non-Euclidean application domains. The main success of GCN, especially in handling dependencies and passing messages within nodes, lies in its approximation to Laplacian smoothing. As a matter of fact, this smoothing technique can not only encourage must-link node pairs to get closer but also push cannot-link pairs to shrink together, which potentially cause serious feature shrink or oversmoothing problem, especially when stacking graph convolution in multiple layers or steps. For learning time-evolving patterns, a natural solution is to preserve historical state and combine it with the current interactions to obtain the most recent representation. Then the serious feature shrink or oversmoothing problem could happen when stacking graph convolution explicitly or implicitly according to current prevalent methods, which would make nodes too similar to distinguish each other. To solve this problem in dynamic graph embedding, we analyze the shrinking properties in the node embedding space at first, and then design a simple yet versatile method, which exploits L2 feature normalization constraint to rescale all nodes to hypersphere of a unit ball so that nodes would not shrink together, and yet similar nodes can still get closer. Extensive experiments on four real-world dynamic graph datasets compared with competitive baseline models demonstrate the effectiveness of the proposed method.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑