Graph neural networks are emerging as continuation of deep learning success w.r.t. graph data. Tens of different graph neural network variants have been proposed, most following a neighborhood aggregation scheme, where the node features are updated via aggregating features of its neighboring nodes from layer to layer. Though related research surges, the power of GNNs are still not on-par-with their counterpart CNNs in computer vision and RNNs in natural language processing. We rethink this problem from the perspective of information propagation, and propose to enhance information propagation among GNN layers by combining heterogeneous aggregations. We argue that as richer information are propagated from shallow to deep layers, the discriminative capability of features formulated by GNN can benefit from it. As our first attempt in this direction, a new generic GNN layer formulation and upon this a new GNN variant referred as HAG-Net is proposed. We empirically validate the effectiveness of HAG-Net on a number of graph classification benchmarks, and elaborate all the design options and criterions along with.
HAGNN: Hybrid Aggregation for Heterogeneous Graph Neural Networks
State Key Laboratory for Novel Software Technology, Nanjing University（南京大学现代软件技术国家重点实验室）；Research Center for Graph Computing, Zhejiang Lab（浙江实验室图计算研究中心）
Virtual Nodes based Heterogeneous Graph Convolutional Neural Network for Efficient Long-Range Information Aggregation
School of Statistics and Mathematics, Guangdong University of Finance & Economics（广东金融学院统计与数学学院）；School of Digital Economics, Guangdong University of Finance & Economics（广东金融学院数字经济学学院）
MAPN: Enhancing Heterogeneous Sparse Graph Representation by Mamba-based Asynchronous Aggregation
School of Computer Science, Fudan University（复旦大学计算机科学学院）；School of Software, Fudan University（复旦大学软件学院）；Shanghai Key Laboratory of Data Science（上海数据科学重点实验室）
Zhejiang University（浙江大学）；IBM T. J. Watson Research Center（IBM托马斯·J·沃森研究中心）；JD.COM Silicon Valley Research Center（京东硅谷研发中心）；Squirrel AI Learning（松鼠AI智适应教育）
Crowdsourcing has attracted much attention for its convenience to collect labels from non-expert workers instead of experts. However, due to the high level of noise from the non-experts, an aggregation model that learns the true label by incorporating the source credibility is required. In this paper, we propose a novel framework based on graph neural networks for aggregating crowd labels. We construct a heterogeneous graph between workers and tasks and derive a new graph neural network to learn the representations of nodes and the true labels. Besides, we exploit the unknown latent interactio…
Ant Financial Services Group（蚂蚁金融服务集团）；Georgia Institute of Technology（佐治亚理工学院）
We present, GEM, the first heterogeneous graph neural network approach for detecting malicious accounts at Alipay, one of the world's leading mobile cashless payment platform. Our approach, inspired from a connected subgraph approach, adaptively learns discriminative embeddings from heterogeneous account-device graphs based on two fundamental weaknesses of attackers, i.e. device aggregation and activity aggregation. For the heterogeneous graph consists of various types of nodes, we propose an attention mechanism to learn the importance of different types of nodes, while using the sum operator …
Case Western Reserve University（凯斯西储大学）；University of Notre Dame（圣母大学）；Brandeis University（布兰迪斯大学）；Emory University（埃默里大学）
Graph neural networks (GNNs) have been broadly studied on dynamic graphs for their representation learning, majority of which focus on graphs with homogeneous structures in the spatial domain. However, many real-world graphs - i.e., heterogeneous temporal graphs (HTGs) - evolve dynamically in the context of heterogeneous graph structures. The dynamics associated with heterogeneity have posed new challenges for HTG representation learning. To solve this problem, in this paper, we propose heterogeneous temporal graph neural network (HTGNN) to integrate both spatial and temporal dependencies whil…
Most real-world datasets are inherently heterogeneous graphs, which involve a diversity of node and relation types. Heterogeneous graph embedding is to learn the structure and semantic information from the graph, and then embed it into the low-dimensional node representation. Existing methods usually capture the composite relation of a heterogeneous graph by defining metapath, which represent a semantic of the graph. However, these methods either ignore node attributes, or discard the local and global information of the graph, or only consider one metapath. To address these limitations, we pro…
Graph neural networks (GNNs) have achieved great success in many graph learning tasks. The main aspect powering existing GNNs is the multi-layer network architecture to learn the nonlinear graph representations for the specific learning tasks. The core operation in GNNs is message propagation in which each node updates its representation by aggregating its neighbors' representations. Existing GNNs mainly adopt either linear neighborhood aggregation (mean,sum) or max aggregator in their message propagation. (1) For linear aggregators, the whole nonlinearity and network's capacity of GNNs are ge…
Temporal Aggregation and Propagation Graph Neural Networks for Dynamic Representation
Hangzhou City University（浙大城市学院）；Zhejiang University（浙江大学）；National University of Singapore（新加坡国立大学）；Shanghai Institute for Advanced Study, Zhejiang University（浙江大学上海高等研究院）
A large number of real-world graphs or networks are inherently heterogeneous, involving a diversity of node types and relation types. Heterogeneous graph embedding is to embed rich structural and semantic information of a heterogeneous graph into low-dimensional node representations. Existing models usually define multiple metapaths in a heterogeneous graph to capture the composite relations and guide neighbor selection. However, these models either omit node content features, discard intermediate nodes along the metapath, or only consider one metapath. To address these three limitations, we p…
Recently, graph neural networks have shown the superiority of modeling the complex topological structures in heterogeneous network-based recommender systems. Due to the diverse interactions among nodes and abundant semantics emerging from diverse types of nodes and edges, there is a bursting research interest in learning expressive node representations in multiplex heterogeneous networks. One of the most important tasks in recommender systems is to predict the potential connection between two nodes under a specific edge type (i.e., relationship). Although existing studies utilize explicit meta…
Clova AI Research, NAVER Corp.（NAVER公司Clova人工智能研究院）；Search Solution Inc.（搜索解决方案公司）；LINE Plus Corp.（LINE Plus公司）
Graph Neural Networks (GNNs) have been emerging as a promising method for relational representation including recommender systems. However, various challenging issues of social graphs hinder the practical usage of GNNs for social recommendation, such as their complex noisy connections and high heterogeneity. The oversmoothing of GNNs is an obstacle of GNN-based social recommendation as well. Here we propose a new graph embedding method Heterogeneous Graph Propagation (HGP) to tackle these issues. HGP uses a group-user-item tripartite graph as input to reduce the number of edges and the complex…
FedTSA: A Cluster-based Two-Stage Aggregation Method for Model-heterogeneous Federated Learning
University of Helsinki（赫尔辛基大学）；Zhejiang University（浙江大学）；Simon Fraser University（西蒙菲莎大学）；Norwegian University of Science and Technology（挪威科技大学）；The Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州））
FedSKD: Aggregation-free Model-heterogeneous Federated Learning via Multi-dimensional Similarity Knowledge Distillation for Medical Image Classification
A Comparative Study of Federated Learning Aggregation Strategies under Homogeneous and Heterogeneous Data Distributions
School of Electrical and Computer Engineering, National Technical University of Athens, Greece（电气与计算机工程学院，国家技术大学亚历山大学院，希腊）；Department of Informatics and Telematics, Harokopio University of Athens, Greece（信息与电信学院，哈罗基奥大学亚历山大学院，希腊）
FedSCAM (Federated Sharpness-Aware Minimization with Clustered Aggregation and Modulation): Scam-resistant SAM for Robust Federated Optimization in Heterogeneous Environments
Department of Computer Science, Lahore University of Management Sciences, Lahore, Pakistan（计算机科学系，拉合尔管理科学大学，拉合尔，巴基斯坦）
Decentralized Federated Learning for Heterogeneous Multi-Task Semantic Communication
School of Information and Communication Engineering, Beijing University of Posts and Telecommunications (BUPT)（北京邮电大学信息与通信工程学院）；Beijing Information Science and Technology University（北京信息科技大学）
FedHL: Federated Learning for Heterogeneous Low-Rank Adaptation via Unbiased Aggregation
Institute of Artificial Intelligence and Future Networks, Beijing Normal University, Zhuhai（人工智能与未来网络研究院，北京师范大学，珠海）；School of Computer Science and Artificial Intelligence, Zhengzhou University（计算机科学与人工智能学院，郑州大学）；School of Software, Nanchang University（软件学院，南昌大学）
University of Virginia（弗吉尼亚大学）；University of California, Los Angeles（加州大学洛杉矶分校）；California Institute of Technology（加州理工学院）
In heterogeneous rank aggregation problems, users often exhibit various accuracy levels when comparing pairs of items. Thus a uniform querying strategy over users may not be optimal. To address this issue, we propose an elimination-based active sampling strategy, which estimates the ranking of items via noisy pairwise comparisons from users and improves the users' average accuracy by maintaining an active set of users. We prove that our algorithm can return the true ranking of items with high probability. We also provide a sample complexity bound for the proposed algorithm which is better than…
FedAPA: Server-side Gradient-Based Adaptive Personalized Aggregation for Federated Learning on Heterogeneous Data
College of Information Science and Technology, Jinan University（暨南大学信息科学技术学院）；College of Information and Computational Science, Jilin University（吉林大学信息与计算科学学院）；University of Toronto（多伦多大学）；The Hong Kong Polytechnic University（香港理工大学）
LAPA-based Dynamic Privacy Optimization for Wireless Federated Learning in Heterogeneous Environments
College of Electronics and Information Engineering, Tongji University（同济大学电子与信息工程学院）；Commonwealth Science and Industrial Research Organization (CSIRO)（共同科学与工业研究组织）；School of Electrical and Information Engineering Faculty of Engineering, The University of Sydney（悉尼大学电气与信息工程学院）
University of Virginia（弗吉尼亚大学）；University of California, Los Angeles（加利福尼亚大学洛杉矶分校）
We propose the Heterogeneous Thurstone Model (HTM) for aggregating ranked data, which can take the accuracy levels of different users into account. By allowing different noise distributions, the proposed HTM model maintains the generality of Thurstone's original framework, and as such, also extends the Bradley-Terry-Luce (BTL) model for pairwise comparisons to heterogeneous populations of users. Under this framework, we also propose a rank aggregation algorithm based on alternating gradient descent to estimate the underlying item scores and accuracy levels of different users simultaneously fro…
Sepsis is a life-threatening condition that seriously endangers millions of people over the world. Hopefully, with the widespread availability of electronic health records (EHR), predictive models that can effectively deal with clinical sequential data increase the possibility to predict sepsis and take early preventive treatment. However, the early prediction is challenging because patients' sequential data in EHR contains temporal interactions of multiple clinical events. And capturing temporal interactions in the long event sequence is hard for traditional LSTM. Rather than directly applyin…
On the Power of Adaptive Weighted Aggregation in Heterogeneous Federated Learning and Beyond
University of Electronic Science and Technology of China（电子科技大学）；Fudan University（复旦大学）；Harbin Institute of Technology Shenzhen（哈尔滨工业大学（深圳））；Meta AI；Chinese University of Hong Kong Shenzhen（香港中文大学（深圳））
Although federated learning has achieved many breakthroughs recently, the heterogeneous nature of the learning environment greatly limits its performance and hinders its real-world applications. The heterogeneous data, time-varying wireless conditions and computing-limited devices are three main challenges, which often result in an unstable training process and degraded accuracy. Herein, we propose strategies to address these challenges. Targeting the heterogeneous data distribution, we propose a novel adaptive mixing aggregation (AMA) scheme that mixes the model updates from previous rounds w…
XAI-SOH-FL: Enhancing SOH-FL with Adaptive Aggregation and Explainable AI for Intrusion Detection in Heterogeneous IoT
School of Electrical Engineering and Computer Science (SEECS), National University of Sciences and Technology (NUST)（电气工程与计算机科学学院（SEECS），国家 sciences and Technology（NUST））
Iowa State University（爱荷华州立大学）；IBM Almaden Research Center（IBM阿尔马登研究中心）
Federated learning~(FL) facilitates the training and deploying AI models on edge devices. Preserving user data privacy in FL introduces several challenges, including expensive communication costs, limited resources, and data heterogeneity. In this paper, we propose SPATL, an FL method that addresses these issues by: (a) introducing a salient parameter selection agent and communicating selected parameters only; (b) splitting a model into a shared encoder and a local predictor, and transferring its knowledge to heterogeneous clients via the locally customized predictor. Additionally, we leverage…
EASTER: Embedding Aggregation-based Heterogeneous Models Training in Vertical Federated Learning
Beijing Institute of Technology（北京理工大学）；School of Cyberspace Science and Technology（信息空间科学与技术学院）；Key Laboratory of Ethnic Language Intelligent Analysis and Security Governance of MOE（教育部民族语言智能分析与安全治理重点实验室）；Minzu University of China（中央民族大学）；School of Information Engineering（信息工程学院）；School of Computing and Communications（计算与通信学院）
Technical University of Munich（慕尼黑工业大学）；Johns Hopkins University（约翰斯·霍普金斯大学）
Saliency methods interpret the prediction of a neural network by showing the importance of input elements for that prediction. A popular family of saliency methods utilize gradient information. In this work, we empirically show that two approaches for handling the gradient information, namely positive aggregation, and positive propagation, break these methods. Though these methods reflect visually salient information in the input, they do not explain the model prediction anymore as the generated saliency maps are insensitive to the predicted output and are insensitive to model parameter random…
Beijing Advanced Innovation Center for Future Blockchain and Privacy Computing, School of Artificial Intelligence, Beihang University（北京未来区块链与隐私计算先进创新中心，人工智能学院，北京航空航天大学）；Tsinghua University（清华大学）
针对准则奖励中因忽略准则间依赖关系导致的虚假信用传播问题，提出概率图框架Graphical Event Aggregation for Rubric rewards (GEAR)，通过建模潜在伯努利事件和软抑制传播实现依赖感知的奖励聚合，在多个基准上提升性能并减少信用泄漏。
From Isolation to Collaboration: Federated Class-Heterogeneous Learning for Chest X-Ray Classification
University of Maryland School of Medicine（马里兰大学医学院）；University of Maryland Institute for Health Computing（马里兰大学健康计算研究所）；St. Jude Children’s Research Hospital（圣犹大儿童研究医院）
We propose a novel framework to study asynchronous federated learning optimization with delays in gradient updates. Our theoretical framework extends the standard FedAvg aggregation scheme by introducing stochastic aggregation weights to represent the variability of the clients update time, due for example to heterogeneous hardware capabilities. Our formalism applies to the general federated setting where clients have heterogeneous datasets and perform at least one step of stochastic gradient descent (SGD). We demonstrate convergence for such a scheme and provide sufficient conditions for the …
Edge computing has revolutionized the world of mobile and wireless networks world thanks to its flexible, secure, and performing characteristics. Lately, we have witnessed the increasing use of it to make more performing the deployment of machine learning (ML) techniques such as federated learning (FL). FL was debuted to improve communication efficiency compared to conventional distributed machine learning (ML). The original FL assumes a central aggregation server to aggregate locally optimized parameters and might bring reliability and latency issues. In this paper, we conduct an in-depth stu…
Sun Yat-Sen University（中山大学）；Key Laboratory of Machine Intelligence and Advanced Computing, Ministry of Education（教育部机器智能与先进计算重点实验室）；Engineering Research Center for Advanced Computing Engineering Software of Ministry of Education（教育部先进计算工程软件工程研究中心）；GuangDong Province Key Laboratory of Information Security Technology（广东省信息安全技术重点实验室）；the University of Sydney（悉尼大学）
Metro origin-destination prediction is a crucial yet challenging time-series analysis task in intelligent transportation systems, which aims to accurately forecast two specific types of cross-station ridership, i.e., Origin-Destination (OD) one and Destination-Origin (DO) one. However, complete OD matrices of previous time intervals can not be obtained immediately in online metro systems, and conventional methods only used limited information to forecast the future OD and DO ridership separately. In this work, we proposed a novel neural network module termed Heterogeneous Information Aggregati…
A Semi-Supervised Federated Learning Framework with Hierarchical Clustering Aggregation for Heterogeneous Satellite Networks
School of Computer Science and Artificial Intelligence, Wuhan University of Technology（计算机科学与人工智能学院，武汉科技大学）；Hubei Key Laboratory of Transportation Internet of Things, Wuhan University of Technology（交通物联网湖北省重点实验室，武汉科技大学）；School of Engineering, Swinburne University of Technology（工程学院，斯威本科技大学）；School of Computer Science and Technology, Tongji University（计算机科学与技术学院，同济大学）；State Key Laboratory for Novel Software Technology, Nanjing University（新型软件技术国家重点实验室，南京大学）
