Hypergraph neural networks have been promising tools for handling learning tasks involving higher-order data, with notable applications in web graphs, such as modeling multi-way hyperlink structures and complex user interactions. Yet, their generalization abilities in theory are less clear to us. In this paper, we seek to develop margin-based generalization bounds for four representative classes of hypergraph neural networks, including convolutional-based methods (UniGCN), set-based aggregation (AllDeepSets), invariant and equivariant transformations (M-IGN), and tensor-based approaches (T-MPHN). Through the PAC-Bayes framework, our results reveal the manner in which hypergraph structure and spectral norms of the learned weights can affect the generalization bounds, where the key technical challenge lies in developing new perturbation analysis for hypergraph neural networks, which offers a rigorous understanding of how variations in the model's weights and hypergraph structure impact its generalization behavior. Our empirical study examines the relationship between the practical performance and theoretical bounds of the models over synthetic and real-world datasets. One of our primary observations is the strong correlation between the theoretical bounds and empirical loss, with statistically significant consistency in most cases.
Inspired by recent interests of developing machine learning and data mining algorithms on hypergraphs, we investigate in this paper the semi-supervised learning algorithm of propagating "soft labels" (e.g. probability distributions, class membership scores) over hypergraphs, by means of optimal transportation. Borrowing insights from Wasserstein propagation on graphs [Solomon et al. 2014], we re-formulate the label propagation procedure as a message-passing algorithm, which renders itself naturally to a generalization applicable to hypergraphs through Wasserstein barycenters. Furthermore, in a…
Stability and Generalization of Hypergraph Collaborative Networks
Institute of Data Science, The University of Hong Kong（香港大学数据科学学院）；College of Information Science and Technology, Jinan University（暨南大学信息科学技术学院）；The University of Hong Kong（香港大学）
Hypergraph representations are both more efficient and better suited to describe data characterized by relations between two or more objects. In this work, we present a new graph neural network based on message passing capable of processing hypergraph-structured data. We show that the proposed model defines a design space for neural network models for hypergraphs, thus generalizing existing models for hypergraphs. We report experiments on a benchmark dataset for node classification, highlighting the effectiveness of the proposed model with respect to other state-of-the-art methods for graphs a…
Many problems in computer vision and machine learning can be cast as learning on hypergraphs that represent higher-order relations. Recent approaches for hypergraph learning extend graph neural networks based on message passing, which is simple yet fundamentally limited in modeling long-range dependencies and expressive power. On the other hand, tensor-based equivariant neural networks enjoy maximal expressiveness, but their application has been limited in hypergraphs due to heavy computation and strict assumptions on fixed-order hyperedges. We resolve these problems and present Equivariant Hy…
Ho Chi Minh City University of Technology (HCMUT)（胡志明市理工大学）；Vietnam National University Ho Chi Minh City (VNU-HCM)（越南胡志明市国家大学）
This paper constitutes the novel hypergraph convolutional neural networkbased clustering technique. This technique is employed to solve the clustering problem for the Citeseer dataset and the Cora dataset. Each dataset contains the feature matrix and the incidence matrix of the hypergraph (i.e., constructed from the feature matrix). This novel clustering method utilizes both matrices. Initially, the hypergraph auto-encoders are employed to transform both the incidence matrix and the feature matrix from high dimensional space to low dimensional space. In the end, we apply the k-means clustering…
Tsinghua-Berkeley Shenzhen Institute, Tsinghua University（清华大学清华-伯克利深圳学院）
In recent years, hypergraph learning has attracted great attention due to its capacity in representing complex and high-order relationships. However, current neural network approaches designed for hypergraphs are mostly shallow, thus limiting their ability to extract information from high-order neighbors. In this paper, we show both theoretically and empirically, that the performance of hypergraph neural networks does not improve as the number of layers increases, which is known as the over-smoothing problem. To avoid this issue, we develop a new deep hypergraph convolutional network called De…
Hypergraph Neural Diffusion: A PDE-Inspired Framework for Hypergraph Message Passing
School of Mathematics and Statistics, Shandong University（山东大学数学与统计学院）；Academy of Mathematics and Systems Science, Chinese Academy of Sciences（中国科学院数学与系统科学研究院）；School of Mathematical Sciences, University of Chinese Academy of Sciences（中国科学院大学数学科学学院）；Institute of Information Engineering, Chinese Academy of Sciences（中国科学院信息工程研究所）；School of Cyber Security, University of Chinese Academy of Sciences（中国科学院大学网络空间安全学院）
HWL-HIN: A Hypergraph-Level Hypergraph Isomorphism Network as Powerful as the Hypergraph Weisfeiler-Lehman Test with Application to Higher-Order Network Robustness
Beyond Convolution: Advancing Hypergraph Neural Networks with Hypergraph U-Nets
Institute for Financial Services Analytics, University of Delaware（特拉华大学金融服务分析研究所）；Department of Applied Economics and Statistics, University of Delaware（特拉华大学应用经济学与统计学系）；Department of Electrical and Computer Engineering, University of Kentucky（肯塔基大学电气与计算机工程系）；Department of Electrical and Computer Engineering, University of Delaware（特拉华大学电气与计算机工程系）
From Diffusion to Reaction-Diffusion: A Dynamical-Systems View of Oversmoothing in Hypergraph Neural Networks
School of Mathematics and Statistics, Shandong University, Weihai（山东大学（威海）数学与统计学院）；Academy of Mathematics and Systems Science, Chinese Academy of Sciences（中国科学院数学与系统科学研究院）；University of Chinese Academy of Sciences（中国科学院大学）
Ho Chi Minh City University of Technology (HCMUT), VNU-HCM（胡志明市理工大学（HCMUT），越南国家大学胡志明市分校）；EPHE-PSL University（巴黎高等研究实践学院-巴黎文理研究大学）
This paper presents a novel version of the hypergraph neural network method. This method is utilized to solve the noisy label learning problem. First, we apply the PCA dimensional reduction technique to the feature matrices of the image datasets in order to reduce the "noise" and the redundant features in the feature matrices of the image datasets and to reduce the runtime constructing the hypergraph of the hypergraph neural network method. Then, the classic graph-based semi-supervised learning method, the classic hypergraph based semi-supervised learning method, the graph neural network, the …
HyperSAT: Unsupervised Hypergraph Neural Networks for Weighted MaxSAT Problems
School of Mathematical Sciences, University of Chinese Academy of Science（中国科学院大学数学科学学院）；Zhongguancun Laboratory（中关村实验室）；School of Automation Science and Electrical Engineering, Beihang University（北京航空航天大学自动化科学与电气工程学院）
University of Illinois, Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）
Hypergraphs are used to model higher-order interactions amongst agents and there exist many practically relevant instances of hypergraph datasets. To enable efficient processing of hypergraph-structured data, several hypergraph neural network platforms have been proposed for learning hypergraph properties and structure, with a special focus on node classification. However, almost all existing methods use heuristic propagation rules and offer suboptimal performance on many datasets. We propose AllSet, a new hypergraph neural network paradigm that represents a highly general framework for (hyper…
Tencent AI Lab（腾讯人工智能实验室）；Shenzhen International Graduate School, Tsinghua University（清华大学深圳国际研究生院）；University of Texas at Arlington（德克萨斯大学阿灵顿分校）
As a powerful tool for modeling complex relationships, hypergraphs are gaining popularity from the graph learning community. However, commonly used frameworks in deep hypergraph learning focus on hypergraphs with edge-independent vertex weights (EIVWs), without considering hypergraphs with edge-dependent vertex weights (EDVWs) that have more modeling power. To compensate for this, we present General Hypergraph Spectral Convolution (GHSC), a general learning framework that not only handles EDVW and EIVW hypergraphs, but more importantly, enables theoretically explicitly utilizing the existing p…
The Chinese University of Hong Kong（香港中文大学）；Shenzhen Institutes of Advanced Technology（深圳先进技术研究院）；Huawei Noah’s Ark Lab（华为诺亚方舟实验室）；Huawei Hisilicon（华为海思）；Peking University（北京大学）
Precise congestion prediction from a placement solution plays a crucial role in circuit placement. This work proposes the lattice hypergraph (LH-graph), a novel graph formulation for circuits, which preserves netlist data during the whole learning process, and enables the congestion information propagated geometrically and topologically. Based on the formulation, we further developed a heterogeneous graph neural network architecture LHNN, jointing the routing demand regression to support the congestion spot classification. LHNN constantly achieves more than 35% improvements compared with U-net…
Ada-HGNN: Adaptive Sampling for Scalable Hypergraph Neural Networks
University of Amsterdam（阿姆斯特丹大学）；Qualcomm AI Research（高通人工智能研究）；University of Johannesburg（约翰内斯堡大学）；College of Business and Economics, University of Johannesburg（约翰内斯堡大学商学院与经济学院）
University of Illinois at Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）；Amazon（亚马逊）
Despite the prevalence of hypergraphs in a variety of high-impact applications, there are relatively few works on hypergraph representation learning, most of which primarily focus on hyperlink prediction, often restricted to the transductive learning setting. Among others, a major hurdle for effective hypergraph representation learning lies in the label scarcity of nodes and/or hyperedges. To address this issue, this paper presents an end-to-end, bi-level pre-training strategy with Graph Neural Networks for hypergraphs. The proposed framework named HyperGene bears three distinctive advantages.…
University of Texas at Austin（德克萨斯大学奥斯汀分校）；University of Waterloo（滑铁卢大学）；Purdue University（普渡大学）；Georgia Tech（佐治亚理工学院）
Hypergraph neural networks (HNNs) using neural networks to encode hypergraphs provide a promising way to model higher-order relations in data and further solve relevant prediction tasks built upon such higher-order relations. However, higher-order relations in practice contain complex patterns and are often highly irregular. So, it is often challenging to design an HNN that suffices to express those relations while keeping computational efficiency. Inspired by hypergraph diffusion algorithms, this work proposes a new HNN architecture named ED-HNN, which provably represents any continuous equiv…
To deal with irregular data structure, graph convolution neural networks have been developed by a lot of data scientists. However, data scientists just have concentrated primarily on developing deep neural network method for un-directed graph. In this paper, we will present the novel neural network method for directed hypergraph. In the other words, we will develop not only the novel directed hypergraph neural network method but also the novel directed hypergraph based semi-supervised learning method. These methods are employed to solve the node classification task. The two datasets that are u…
This extended abstract describes a framework for analyzing the expressiveness, learning, and (structural) generalization of hypergraph neural networks (HyperGNNs). Specifically, we focus on how HyperGNNs can learn from finite datasets and generalize structurally to graph reasoning problems of arbitrary input sizes. Our first contribution is a fine-grained analysis of the expressiveness of HyperGNNs, that is, the set of functions that they can realize. Our result is a hierarchy of problems they can solve, defined in terms of various hyperparameters such as depths and edge arities. Next, we anal…
In this work, we introduce a hypergraph representation learning framework called Hypergraph Neural Networks (HNN) that jointly learns hyperedge embeddings along with a set of hyperedge-dependent embeddings for each node in the hypergraph. HNN derives multiple embeddings per node in the hypergraph where each embedding for a node is dependent on a specific hyperedge of that node. Notably, HNN is accurate, data-efficient, flexible with many interchangeable components, and useful for a wide range of hypergraph learning tasks. We evaluate the effectiveness of the HNN framework for hyperedge predict…
Institute of Image Processing and Pattern Recognition, Shanghai Jiao Tong University（上海交通大学图像处理与模式识别研究所）
Hypergraph, an expressive structure with flexibility to model the higher-order correlations among entities, has recently attracted increasing attention from various research domains. Despite the success of Graph Neural Networks (GNNs) for graph representation learning, how to adapt the powerful GNN-variants directly into hypergraphs remains a challenging problem. In this paper, we propose UniGNN, a unified framework for interpreting the message passing process in graph and hypergraph neural networks, which can generalize general GNN models into hypergraphs. In this framework, meticulously-desi…
Hyperbolic Hypergraph Neural Networks for Multi-Relational Knowledge Hypergraph Representation
National Engineering Research Center for Big Data Technology and System（国家大数据技术与系统工程研究中心）；School of Computer Science and Technology（计算机科学与技术学院）；Huazhong University of Science and Technology（华中科技大学）
Graph representation learning for hypergraphs can be used to extract patterns among higher-order interactions that are critically important in many real world problems. Current approaches designed for hypergraphs, however, are unable to handle different types of hypergraphs and are typically not generic for various learning tasks. Indeed, models that can predict variable-sized heterogeneous hyperedges have not been available. Here we develop a new self-attention based graph neural network called Hyper-SAGNN applicable to homogeneous and heterogeneous hypergraphs with variable hyperedge sizes. …
University of Amsterdam（阿姆斯特丹大学）；Indian Institute of Technology Dhanbad（印度理工学院丹巴德分校）；AIQ
Graphs are the most ubiquitous data structures for representing relational datasets and performing inferences in them. They model, however, only pairwise relations between nodes and are not designed for encoding the higher-order relations. This drawback is mitigated by hypergraphs, in which an edge can connect an arbitrary number of nodes. Most hypergraph learning approaches convert the hypergraph structure to that of a graph and then deploy existing geometric deep learning methods. This transformation leads to information loss, and sub-optimal exploitation of the hypergraph's expressive power…
Hypergraph Neural Sheaf Diffusion: A Symmetric Simplicial Set Framework for Higher-Order Learning
POSTECH；Center for Geometry and Physics, Institute for Basic Science (IBS)（几何与物理中心，基础科学研究所）；Korea Advanced Institute of Science and Technology (KAIST)（韩国科学技术院）
Recently, graph neural networks have attracted great attention and achieved prominent performance in various research fields. Most of those algorithms have assumed pairwise relationships of objects of interest. However, in many real applications, the relationships between objects are in higher-order, beyond a pairwise formulation. To efficiently learn deep embeddings on the high-order graph-structured data, we introduce two end-to-end trainable operators to the family of graph neural networks, i.e., hypergraph convolution and hypergraph attention. Whilst hypergraph convolution defines the basi…
Tackling Over-smoothing on Hypergraphs: A Ricci Flow-guided Neural Diffusion Approach
Academy of Mathematics and Systems Science, Chinese Academy of Sciences（中国科学院数学与系统科学研究院）；University of Chinese Academy of Sciences（中国科学院大学）；School of Mathematics and Statistics, Shandong University（山东大学数学与统计学院）；School of Artificial Intelligence, Beihang University（北航人工智能学院）；School of Mathematics, Shandong University（山东大学数学学院）
In this paper, we present a hypergraph neural networks (HGNN) framework for data representation learning, which can encode high-order data correlation in a hypergraph structure. Confronting the challenges of learning representation for complex data in real practice, we propose to incorporate such data structure in a hypergraph, which is more flexible on data modeling, especially when dealing with complex data. In this method, a hyperedge convolution operation is designed to handle the data correlation during representation learning. In this way, traditional hypergraph learning procedure can be…
Towards Trustworthy Hypergraph Neural Networks under Label Noise
Academy of Mathematics and Systems Science, Chinese Academy of Sciences（中国科学院数学与系统科学研究院）；University of Chinese Academy of Sciences（中国科学院大学）；School of Mathematics and Statistics, Shandong University（山东大学数学与统计学院）；School of Artificial Intelligence, Beihang University（北京航空航天大学人工智能学院）
Counterfactual Explanations for Hypergraph Neural Networks
Department of Computer Control and Management Engineering, Sapienza University（计算机控制与管理工程系，萨皮恩扎大学）；Department of Computer Science, Sapienza University（计算机科学系，萨皮恩扎大学）
