GCN for HIN via Implicit Utilization of Attention and Meta-paths
Tianjin University（天津大学）；Emory University（埃默里大学）；University of Illinois at Chicago（伊利诺伊大学芝加哥分校）；University of Illinois at Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）
Di Jin, Zhizhi Yu, Dongxiao He, Carl Yang, Philip S. Yu, Jiawei Han
Heterogeneous information network (HIN) embedding, aiming to map the structure and semantic information in a HIN to distributed representations, has drawn considerable research attention. Graph neural networks for HIN embeddings typically adopt a hierarchical attention (including node-level and meta-path-level attentions) to capture the information from meta-path-based neighbors. However, this complicated attention structure often cannot achieve the function of selecting meta-paths due to severe overfitting. Moreover, when propagating information, these methods do not distinguish direct (one-hop) meta-paths from indirect (multi-hop) ones. But from the perspective of network science, direct relationships are often believed to be more essential, which can only be used to model direct information propagation. To address these limitations, we propose a novel neural network method via implicitly utilizing attention and meta-paths, which can relieve the severe overfitting brought by the current over-parameterized attention mechanisms on HIN. We first use the multi-layer graph convolutional network (GCN) framework, which performs a discriminative aggregation at each layer, along with stacking the information propagation of direct linked meta-paths layer-by-layer, realizing the function of attentions for selecting meta-paths in an indirect way. We then give an effective relaxation and improvement via introducing a new propagation operation which can be separated from aggregation. That is, we first model the whole propagation process with well-defined probabilistic diffusion dynamics, and then introduce a random graph-based constraint which allows it to reduce noise with the increase of layers. Extensive experiments demonstrate the superiority of the new approach over state-of-the-art methods.
MQ-GNN: A Multi-Queue Pipelined Architecture for Scalable and Efficient GNN Training
Department of Computer Science and Engineering, Kyung Hee University (Global Campus), Republic of Korea（计算机科学与工程系，庆熙大学（全球校区），大韩民国）；College of Software, Kyung Hee University (Global Campus), Republic of Korea（软件学院，庆熙大学（全球校区），大韩民国）
BLISS: Bandit Layer Importance Sampling Strategy for Efficient Training of Graph Neural Networks
Wilfrid Laurier University（威尔弗里德·劳里埃大学）；Waterloo, Ontario, Canada（加拿大安大略省温哥华）；Singapore Management University（新加坡管理学院）；Georgia Institute of Technology（佐治亚理工学院）
On the Bottleneck of Graph Neural Networks and its Practical Implications
Technion（以色列理工学院）
自从Gori et al.（2005）和Scarselli et al.（2008）提出图神经网络（GNN）以来，训练GNN的主要问题之一是它们难以在图中遥远的节点之间传播信息。我们对这个问题提出了一个新的解释：GNN在聚合长路径上的消息时容易受到瓶颈的影响。这个瓶颈会导致指数增长的信息过度压缩到固定大小的向量中。因此，GNN无法传播来自远程节点的消息，并且在预测任务依赖于远程交互时表现不佳。在本文中，我们强调了GNN中的过度挤压的固有问题：我们证明了瓶颈阻碍了流行的GNN拟合训练数据中的长距离信号;我们进一步表明，平均吸收传入边缘的GNN，如GCN和GIN，比GAT和GGNN更容易受到过度挤压的影响;最后，我们表明，以前的工作，广泛调整GNN模型的长期问题，遭受过度挤压，打破瓶颈，提高了他们的国家的最先进的结果，没有任何调整或额外的权重。我们的代码可在https://github.com/tech-srl/bottleneck/上获得。
Knowledge-Based Distant Regularization in Learning Probabilistic Models
RIKEN Center for Advanced Intelligence Project（理化学研究所先进智能项目中心）；Security Research Laboratories, NEC Corporation（日本电气股份有限公司安全研究所）；University of Tokyo（东京大学）
Fine-grained Event Categorization with Heterogeneous Graph Convolutional Networks
Beihang University（北京航空航天大学）；Hong Kong University of Science and Technology（香港科技大学）；Tencent（腾讯）；Tsinghua University（清华大学）；University of Illinois at Chicago（伊利诺伊大学芝加哥分校）
Streaming Social Event Detection and Evolution Discovery in Heterogeneous Information Networks
Beihang University（北京航空航天大学）；Hong Kong University of Science and Technology（香港科技大学）；University of Leeds（利兹大学）；Newcastle University（纽卡斯尔大学）；University of Illinois at Chicago（伊利诺伊大学芝加哥分校）；Lehigh University（理海大学）
