Random walks are at the heart of many existing deep learning algorithms for graph data. However, such algorithms have many limitations that arise from the use of random walks, e.g., the features resulting from these methods are unable to transfer to new nodes and graphs as they are tied to node identity. In this work, we introduce the notion of attributed random walks which serves as a basis for generalizing existing methods such as DeepWalk, node2vec, and many others that leverage random walks. Our proposed framework enables these methods to be more widely applicable for both transductive and inductive learning as well as for use on graphs with attributes (if available). This is achieved by learning functions that generalize to new nodes and graphs. We show that our proposed framework is effective with an average AUC improvement of 16.1% while requiring on average 853 times less space than existing methods on a variety of graphs from several domains.
Transformers flexibly operate over sets of real-valued vectors representing task-specific entities and their attributes, where each vector might encode one word-piece token and its position in a sequence, or some piece of information that carries no position at all. But as set processors, transformers are at a disadvantage in reasoning over more general graph-structured data where nodes represent entities and edges represent relations between entities. To address this shortcoming, we generalize transformer attention to consider and update edge vectors in each transformer layer. We evaluate thi…
University of Amsterdam（阿姆斯特丹大学）；The Netherlands（荷兰（注：此处原文本中机构仅明确到University of Amsterdam，The Netherlands为国家，不应作为机构提取，按要求仅提取明确的机构名，故原数据中可能存在格式混淆，实际有效机构为University of Amsterdam））
Graphs are the most ubiquitous form of structured data representation used in machine learning. They model, however, only pairwise relations between nodes and are not designed for encoding the higher-order relations found in many real-world datasets. To model such complex relations, hypergraphs have proven to be a natural representation. Learning the node representations in a hypergraph is more complex than in a graph as it involves information propagation at two levels: within every hyperedge and across the hyperedges. Most current approaches first transform a hypergraph structure to a graph …
SEGNO: Generalizing Equivariant Graph Neural Networks with Physical Inductive Biases
The Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州））；The Hong Kong University of Science and Technology（香港科技大学）；Tencent AI Lab（腾讯AI实验室）
University of Maryland College Park（马里兰大学学院公园分校）；Microsoft Research（微软研究院）
Successor-style representations have many advantages for reinforcement learning: for example, they can help an agent generalize from past experience to new goals, and they have been proposed as explanations of behavioral and neural data from human and animal learners. They also form a natural bridge between model-based and model-free RL methods: like the former they make predictions about future experiences, and like the latter they allow efficient prediction of total discounted rewards. However, successor-style representations are not optimized to generalize across policies: typically, we mai…
Polytechnique Montréal（蒙特利尔理工学院）；Université de Montréal（蒙特利尔大学）；University of Toronto（多伦多大学）
Machine learning (ML) formalizes the problem of getting computers to learn from experience as optimization of performance according to some metric(s) on a set of data examples. This is in contrast to requiring behaviour specified in advance (e.g. by hard-coded rules). Formalization of this problem has enabled great progress in many applications with large real-world impact, including translation, speech recognition, self-driving cars, and drug discovery. But practical instantiations of this formalism make many assumptions - for example, that data are i.i.d.: independent and identically distrib…
Renmin University of China（中国人民大学）；Alibaba Group（阿里巴巴集团）
Graph Neural Networks (GNNs) have received extensive research attention for their promising performance in graph machine learning. Despite their extraordinary predictive accuracy, existing approaches, such as GCN and GPRGNN, are not robust in the face of homophily changes on test graphs, rendering these models vulnerable to graph structural attacks and with limited capacity in generalizing to graphs of varied homophily levels. Although many methods have been proposed to improve the robustness of GNN models, most of these techniques are restricted to the spatial domain and employ complicated de…
This paper reviews graph convolutional neural networks (GCNNs) through the lens of edge-variant graph filters. The edge-variant graph filter is a finite order, linear, and local recursion that allows each node, in each iteration, to weigh differently the information of its neighbors. By exploiting this recursion, we formulate a general framework for GCNNs which considers state-of-the-art solutions as particular cases. This framework results useful to i) understand the tradeoff between local detail and the number of parameters of each solution and ii) provide guidelines for developing a myriad …
Generalizing diffuse interface methods on graphs: non-smooth potentials and hypergraphs
Max Planck Institute for Dynamics of Complex Technical Systems（马克斯·普朗克复杂技术系统动力学研究所）；Department of Computer Science, The University of British Columbia（不列颠哥伦比亚大学计算机科学系）
Advanced methods of applying deep learning to structured data such as graphs have been proposed in recent years. In particular, studies have focused on generalizing convolutional neural networks to graph data, which includes redefining the convolution and the downsampling (pooling) operations for graphs. The method of generalizing the convolution operation to graphs has been proven to improve performance and is widely used. However, the method of applying downsampling to graphs is still difficult to perform and has room for improvement. In this paper, we propose a graph pooling method based on…
Theoretical guarantees for the advantage of GNNs over NNs in generalizing bandlimited functions on Euclidean cubes
Faculty of Mathematics, University of Vienna（维也纳大学数学学院）；Department of Mathematics, Michigan State University（密歇根州立大学数学系）；Department of Statistics and Probability, Michigan State University（密歇根州立大学统计与概率系）
Drawing on some recent results that provide the formalism necessary to definite stationarity for infinite random graphs, this paper initiates the study of statistical and learning questions pertaining to these objects. Specifically, a criterion for the existence of a consistent test for complex hypotheses is presented, generalizing the corresponding results on time series. As an application, it is shown how one can test that a tree has the Markov property, or, more generally, to estimate its memory.
