Node embeddings have become an ubiquitous technique for representing graph data in a low dimensional space. Graph autoencoders, as one of the widely adapted deep models, have been proposed to learn graph embeddings in an unsupervised way by minimizing the reconstruction error for the graph data. However, its reconstruction loss ignores the distribution of the latent representation, and thus leading to inferior embeddings. To mitigate this problem, we propose a random walk based method to regularize the representations learnt by the encoder. We show that the proposed novel enhancement beats the existing state-of-the-art models by a large margin (upto 7.5\%) for node clustering task, and achieves state-of-the-art accuracy on the link prediction task for three standard datasets, cora, citeseer and pubmed. Code available at https://github.com/MysteryVaibhav/DW-GAE.
Graph neural networks have been used for a variety of learning tasks, such as link prediction, node classification, and node clustering. Among them, link prediction is a relatively under-studied graph learning task, with current state-of-the-art models based on one- or two-layer of shallow graph auto-encoder (GAE) architectures. In this paper, we focus on addressing a limitation of current methods for link prediction, which can only use shallow GAEs and variational GAEs, and creating effective methods to deepen (variational) GAE architectures to achieve stable and competitive performance. Our …
Missing data imputation (MDI) is crucial when dealing with tabular datasets across various domains. Autoencoders can be trained to reconstruct missing values, and graph autoencoders (GAE) can additionally consider similar patterns in the dataset when imputing new values for a given instance. However, previously proposed GAEs suffer from scalability issues, requiring the user to define a similarity metric among patterns to build the graph connectivity beforehand. In this paper, we leverage recent progress in latent graph imputation to propose a novel EdGe Generation Graph AutoEncoder (EGG-GAE) …
HC-GAE: The Hierarchical Cluster-based Graph Auto-Encoder for Graph Representation Learning
School of Artificial Intelligence, Beijing Normal University（北京师范大学人工智能学院）；School of Information, Central University of Finance and Economics（中央财经大学信息学院）；Key Laboratory of Intelligent Education Technology and Application of Zhejiang Province, Zhejiang Normal University（浙江师范大学浙江省智能教育技术与应用重点实验室）；Department of Computer Science, University of York（约克大学计算机学院）
Norm Augmented Graph AutoEncoders for Link Prediction
State Key Laboratory for Novel Software Technology, Nanjing University（南京大学现代软件技术国家重点实验室）；Tsinghua Shenzhen International Graduate School, Tsinghua University（清华大学深圳国际研究生院）；School of Electrical Engineering and Computer Science, The University of Queensland（昆士兰大学电气工程与计算机科学学院）
ML Collective（机器学习联盟）；National Institute of Science Education and Research, Bhubaneswar（布巴内斯瓦尔国家科学教育与研究学院）；Homi Bhabha National Institute, Mumbai（孟买霍米·巴巴国立研究所）；M S Ramaiah Institute of Technology（M S Ramaiah理工学院）
Lithium-ion batteries are powering the ongoing transportation electrification revolution. Lithium-ion batteries possess higher energy density and favourable electrochemical properties which make it a preferable energy source for electric vehicles. Precise estimation of battery parameters (Charge capacity, voltage etc) is vital to estimate the available range in an electric vehicle. Graph-based estimation techniques enable us to understand the variable dependencies underpinning them to improve estimates. In this paper we employ Graph Neural Networks for battery parameter estimation, we introduc…
Nanjing University（南京大学）；Tsinghua University（清华大学）；ShanghaiTech University（上海科技大学）；Beihang University（北京航空航天大学）；Nanjing University of Information Science and Technology（南京信息工程大学）
Recently, the graph neural network (GNN) has shown great power in matrix completion by formulating a rating matrix as a bipartite graph and then predicting the link between the corresponding user and item nodes. The majority of GNN-based matrix completion methods are based on Graph Autoencoder (GAE), which considers the one-hot index as input, maps a user (or item) index to a learnable embedding, applies a GNN to learn the node-specific representations based on these learnable embeddings and finally aggregates the representations of the target users and its corresponding item nodes to predict …
Graph autoencoders (GAEs) are powerful tools in representation learning for graph embedding. However, the performance of GAEs is very dependent on the quality of the graph structure, i.e., of the adjacency matrix. In other words, GAEs would perform poorly when the adjacency matrix is incomplete or be disturbed. In this paper, two novel unsupervised graph embedding methods, unsupervised graph embedding via adaptive graph learning (BAGE) and unsupervised graph embedding via variational adaptive graph learning (VBAGE) are proposed. The proposed methods expand the application range of GAEs on grap…
Link prediction is one of the key problems for graph-structured data. With the advancement of graph neural networks, graph autoencoders (GAEs) and variational graph autoencoders (VGAEs) have been proposed to learn graph embeddings in an unsupervised way. It has been shown that these methods are effective for link prediction tasks. However, they do not work well in link predictions when a node whose degree is zero (i.g., isolated node) is involved. We have found that GAEs/VGAEs make embeddings of isolated nodes close to zero regardless of their content features. In this paper, we propose a nove…
Deezer Research（Deezer研究院）；École Polytechnique, IP Paris（巴黎综合理工学院，巴黎理工学院）；Harvard University（哈佛大学）
Graph autoencoders (GAE) and variational graph autoencoders (VGAE) emerged as powerful methods for link prediction (LP). Their performances are less impressive on community detection (CD), where they are often outperformed by simpler alternatives such as the Louvain method. It is still unclear to what extent one can improve CD with GAE and VGAE, especially in the absence of node features. It is moreover uncertain whether one could do so while simultaneously preserving good performances on LP in a multi-task setting. In this workshop paper, summarizing results from our journal publication (Salh…
Deezer Research（Deezer研究院）；École Polytechnique（巴黎综合理工学院）；Institut Polytechnique de Paris（巴黎理工学院）；Athens University of Economics and Business（雅典经济与商业大学）
Graph autoencoders (GAE) and variational graph autoencoders (VGAE) emerged as powerful methods for link prediction. Their performances are less impressive on community detection problems where, according to recent and concurring experimental evaluations, they are often outperformed by simpler alternatives such as the Louvain method. It is currently still unclear to which extent one can improve community detection with GAE and VGAE, especially in the absence of node features. It is moreover uncertain whether one could do so while simultaneously preserving good performances on link prediction. I…
Event detection has been an important task in transportation, whose task is to detect points in time when large events disrupts a large portion of the urban traffic network. Travel information {Origin-Destination} (OD) matrix data by map service vendors has large potential to give us insights to discover historic patterns and distinguish anomalies. However, to fully capture the spatial and temporal traffic patterns remains a challenge, yet serves a crucial role for effective anomaly detection. Meanwhile, existing anomaly detection methods have not well-addressed the extreme data sparsity and h…
Adversarial training is an approach for increasing model's resilience against adversarial perturbations. Such approaches have been demonstrated to result in models with feature representations that generalize better. However, limited works have been done on adversarial training of models on graph data. In this paper, we raise such a question { does adversarial training improve the generalization of graph representations. We formulate L2 and L1 versions of adversarial training in two powerful node embedding methods: graph autoencoder (GAE) and variational graph autoencoder (VGAE). We conduct ex…
We present two instances, L-GAE and L-VGAE, of the variational graph auto-encoding family (VGAE) based on separating feature propagation operations from graph convolution layers typically found in graph learning methods to a single linear matrix computation made prior to input in standard auto-encoder architectures. This decoupling enables the independent and fixed design of the auto-encoder without requiring additional GCN layers for every desired increase in the size of a node's local receptive field. Fixing the auto-encoder enables a fairer assessment on the size of a nodes receptive field …
Beijing University of Posts and Telecommunications（北京邮电大学）
With the rapid development of big data, how to efficiently and accurately discover tight community structures in large-scale networks for knowledge discovery has attracted more and more attention. In this paper, a community detection framework based on Graph AutoEncoder Reconstruction (noted as GAER) is proposed for the first time. GAER is a highly scalable framework which does not require any prior information. We decompose the graph autoencoder-based one-step encoding into the two-stage encoding framework to adapt to the real-world big data system by reducing complexity from the original O(N…
Graph clustering, aiming to partition nodes of a graph into various groups via an unsupervised approach, is an attractive topic in recent years. To improve the representative ability, several graph auto-encoder (GAE) models, which are based on semi-supervised graph convolution networks (GCN), have been developed and they achieve good results compared with traditional clustering methods. However, all existing methods either fail to utilize the orthogonal property of the representations generated by GAE, or separate the clustering and the learning of neural networks. We first prove that the rela…
Contributions to Representation Learning with Graph Autoencoders and Applications to Music Recommendation
Institut Polytechnique de Paris（巴黎理工学院）；University of California, Los Angeles（加利福尼亚大学洛杉矶分校）；Tsinghua University（清华大学）；INRIA（法国国家信息与自动化研究所）；DeepMind；Deezer
Document summarization aims to create a precise and coherent summary of a text document. Many deep learning summarization models are developed mainly for English, often requiring a large training corpus and efficient pre-trained language models and tools. However, English summarization models for low-resource Indian languages are often limited by rich morphological variation, syntax, and semantic differences. In this paper, we propose GAE-ISumm, an unsupervised Indic summarization model that extracts summaries from text documents. In particular, our proposed model, GAE-ISumm uses Graph Autoenc…
University of Quebec at Montreal（魁北克大学蒙特利尔分校）；University of Bahrain（巴林大学）
Most recent graph clustering methods have resorted to Graph Auto-Encoders (GAEs) to perform joint clustering and embedding learning. However, two critical issues have been overlooked. First, the accumulative error, inflicted by learning with noisy clustering assignments, degrades the effectiveness and robustness of the clustering model. This problem is called Feature Randomness. Second, reconstructing the adjacency matrix sets the model to learn irrelevant similarities for the clustering task. This problem is called Feature Drift. Interestingly, the theoretical relation between the aforementio…
In computer vision research, the process of automating architecture engineering, Neural Architecture Search (NAS), has gained substantial interest. In the past, NAS was hardly accessible to researchers without access to large-scale compute systems, due to very long compute times for the recurrent search and evaluation of new candidate architectures. The NAS-Bench-101 dataset facilitates a paradigm change towards classical methods such as supervised learning to evaluate neural architectures. In this paper, we propose a graph encoder built upon Graph Neural Networks (GNN). We demonstrate the eff…
SunPower Corporation（SunPower公司）；SolarEdge Technologies（SolarEdge科技公司）；Case Western Reserve University（凯斯西储大学）；C2 Energy Capital（C2能源资本公司）；Sandia National Labs（桑迪亚国家实验室）
The integration of the global Photovoltaic (PV) market with real time data-loggers has enabled large scale PV data analytical pipelines for power forecasting and long-term reliability assessment of PV fleets. Nevertheless, the performance of PV data analysis heavily depends on the quality of PV timeseries data. This paper proposes a novel Spatio-Temporal Denoising Graph Autoencoder (STD-GAE) framework to impute missing PV Power Data. STD-GAE exploits temporal correlation, spatial coherence, and value dependencies from domain knowledge to recover missing data. Experimental results show that STD…
Auto-encoders are perhaps the best-known non-probabilistic methods for representation learning. They are conceptually simple and easy to train. Recent theoretical work has shed light on their ability to capture manifold structure, and drawn connections to density modelling. This has motivated researchers to seek ways of auto-encoder scoring, which has furthered their use in classification. Gated auto-encoders (GAEs) are an interesting and flexible extension of auto-encoders which can learn transformations among different images or pixel covariances within images. However, they have been much l…
State Key Laboratory of Industrial Control and Technology（工业控制技术国家重点实验室）；Zhejiang University（浙江大学）
We consider the problem of image representation for the tasks of unsupervised learning and semi-supervised learning. In those learning tasks, the raw image vectors may not provide enough representation for their intrinsic structures due to their highly dense feature space. To overcome this problem, the raw image vectors should be mapped to a proper representation space which can capture the latent structure of the original data and represent the data explicitly for further learning tasks such as clustering. Inspired by the recent research works on deep neural network and representation learnin…
$\texttt{SynC}$: Synergistic Boosting of Structure and Representation for Deep Graph Clustering
School of Computer Science and Technology/the School of Artificial Intelligence, China University of Mining and Technology（计算机科学与技术学院/人工智能学院，中国矿业大学）；Mine Digitization Engineering Research Center of Ministry of Education（教育部矿山数字化工程研究中心）；College of Intelligence and Computing, Tianjin University（智能与计算学院，天津大学）；Key Laboratory of Knowledge Engineering with Big Data (the Ministry of Education of China), Hefei University of Technology（大数据知识工程重点实验室（教育部），合肥工业大学）
Graph Autoencoders for Embedding Learning in Brain Networks and Major Depressive Disorder Identification
Monash University Malaysia（莫纳什大学马来西亚校区）；Vanderbilt University Medical Center（范德比尔特大学医学中心）；King Abdullah University of Science and Technology（阿卜杜拉国王科技大学）
Graph Convolutional Networks (GCN) have achieved state-of-art results on single text classification tasks like sentiment analysis, emotion detection, etc. However, the performance is achieved by testing and reporting on resource-rich languages like English. Applying GCN for multi-task text classification is an unexplored area. Moreover, training a GCN or adopting an English GCN for Indian languages is often limited by data availability, rich morphological variation, syntax, and semantic differences. In this paper, we study the use of GCN for the Telugu language in single and multi-task setting…
Data-Agnostic Model Poisoning against Federated Learning: A Graph Autoencoder Approach
University of Cambridge（剑桥大学）；Real-Time and Embedded Computing Systems Research Centre (CISTER)（实时与嵌入式计算系统研究中心（CISTER））；CyLab Security and Privacy Institute, Carnegie Mellon University（卡内基梅隆大学CyLab安全与隐私研究所）；Commonwealth Scientific and Industrial Research Organization (CSIRO)（联邦科学与工业研究组织（CSIRO））；Koç University（科奇大学）；Princeton University（普林斯顿大学）
Amirkabir University of Technology（阿米尔卡比尔理工大学）；Iranian Comprehensive Hemophilia Care Center (ICHCC)（伊朗综合血友病护理中心）
Resting-state functional MRI (rs-fMRI) in functional neuroimaging techniques have improved in brain disorders, dysfunction studies via mapping the topology of the brain connections, i.e. connectopic mapping. Since, there are the slight differences between healthy and unhealthy brain regions and functions, investigation into the complex topology of functional and structural brain networks in human is a complicated task with the growth of evaluation criteria. Irregular graph deep learning applications have widely spread to understanding human cognitive functions that are linked to gene expressio…
