Graph neural networks have been widely used on modeling graph data, achieving impressive results on node classification and link prediction tasks. Yet, obtaining an accurate representation for a graph further requires a pooling function that maps a set of node representations into a compact form. A simple sum or average over all node representations considers all node features equally without consideration of their task relevance, and any structural dependencies among them. Recently proposed hierarchical graph pooling methods, on the other hand, may yield the same representation for two different graphs that are distinguished by the Weisfeiler-Lehman test, as they suboptimally preserve information from the node features. To tackle these limitations of existing graph pooling methods, we first formulate the graph pooling problem as a multiset encoding problem with auxiliary information about the graph structure, and propose a Graph Multiset Transformer (GMT) which is a multi-head attention based global pooling layer that captures the interaction between nodes according to their structural dependencies. We show that GMT satisfies both injectiveness and permutation invariance, such that it is at most as powerful as the Weisfeiler-Lehman graph isomorphism test. Moreover, our methods can be easily extended to the previous node clustering approaches for hierarchical graph pooling. Our experimental results show that GMT significantly outperforms state-of-the-art graph pooling methods on graph classification benchmarks with high memory and time efficiency, and obtains even larger performance gain on graph reconstruction and generation tasks.
Graph neural networks have recently achieved remarkable success in representing graph-structured data, with rapid progress in both the node embedding and graph pooling methods. Yet, they mostly focus on capturing information from the nodes considering their connectivity, and not much work has been done in representing the edges, which are essential components of a graph. However, for tasks such as graph reconstruction and generation, as well as graph classification tasks for which the edges are important for discrimination, accurately representing edges of a given graph is crucial to the succe…
GALUX inc.（GALUX公司）；Seoul National University（首尔国立大学）
In drug discovery, aqueous solubility is an important pharmacokinetic property which affects absorption and assay availability of drug. Thus, in silico prediction of solubility has been studied for its utility in virtual screening and lead optimization. Recently, machine learning (ML) methods using experimental data has been popular because physics-based methods like quantum mechanics and molecular dynamics are not suitable for high-throughput tasks due to its computational costs. However, ML method can exhibit over-fitting problem in a data-deficient condition, and this is the case for most c…
Accurate traffic conditions prediction provides a solid foundation for vehicle-environment coordination and traffic control tasks. Because of the complexity of road network data in spatial distribution and the diversity of deep learning methods, it becomes challenging to effectively define traffic data and adequately capture the complex spatial nonlinear features in the data. This paper applies two hierarchical graph pooling approaches to the traffic prediction task to reduce graph information redundancy. First, this paper verifies the effectiveness of hierarchical graph pooling methods in tra…
NEST: Nested Event Stream Transformer for Sequences of Multisets
Department of Biostatistics & Bioinformatics, Duke University（生物统计学与生物信息学系，杜克大学）；Department of Biomedical Engineering, Duke University（生物医学工程系，杜克大学）；Department of Pediatrics, Duke University（儿科学系，杜克大学）
National Institute of Mental Health and Neurosciences (NIMHANS)（国家精神卫生与神经科学研究所（NIMHANS））
Alzheimer's Disease (AD) is the most widespread neurodegenerative disease, affecting over 50 million people across the world. While its progression cannot be stopped, early and accurate diagnostic testing can drastically improve quality of life in patients. Currently, only qualitative means of testing are employed in the form of scoring performance on a battery of cognitive tests. The inherent disadvantage of this method is that the burden of an accurate diagnosis falls on the clinician's competence. Quantitative methods like MRI scan assessment are inaccurate at best,due to the elusive nature…
The University of Sydney（悉尼大学）；Max Planck Institute for Mathematics in the Sciences（马克斯·普朗克科学数学研究所）；Shanghai Jiao Tong University（上海交通大学）；The University of New South Wales（新南威尔士大学）；University of Cambridge（剑桥大学）；Zhejiang Normal University（浙江师范大学）；University of California, Los Angeles（加州大学洛杉矶分校）
This paper presents a new approach for assembling graph neural networks based on framelet transforms. The latter provides a multi-scale representation for graph-structured data. We decompose an input graph into low-pass and high-pass frequencies coefficients for network training, which then defines a framelet-based graph convolution. The framelet decomposition naturally induces a graph pooling strategy by aggregating the graph feature into low-pass and high-pass spectra, which considers both the feature values and geometry of the graph data and conserves the total information. The graph neural…
Global covariance pooling (GCP) aims at exploiting the second-order statistics of the convolutional feature. Its effectiveness has been demonstrated in boosting the classification performance of Convolutional Neural Networks (CNNs). Singular Value Decomposition (SVD) is used in GCP to compute the matrix square root. However, the approximate matrix square root calculated using Newton-Schulz iteration \cite{li2018towards} outperforms the accurate one computed via SVD \cite{li2017second}. We empirically analyze the reason behind the performance gap from the perspectives of data precision and grad…
Homomorphic Encryption (HE) is one of the most promising security solutions to emerging Machine Learning as a Service (MLaaS). Leveled-HE (LHE)-enabled Convolutional Neural Networks (LHECNNs) are proposed to implement MLaaS to avoid large bootstrapping overhead. However, prior LHECNNs have to pay significant computing overhead but achieve only low inference accuracy, due to their polynomial approximation activations and poolings. Stacking many polynomial approximation activation layers in a network greatly reduces inference accuracy, since the polynomial approximation activation errors lead to…
We propose MultiRocket, a fast time series classification (TSC) algorithm that achieves state-of-the-art performance with a tiny fraction of the time and without the complex ensembling structure of many state-of-the-art methods. MultiRocket improves on MiniRocket, one of the fastest TSC algorithms to date, by adding multiple pooling operators and transformations to improve the diversity of the features generated. In addition to processing the raw input series, MultiRocket also applies first order differences to transform the original series. Convolutions are applied to both representations, an…
University of Saskatchewan（萨斯喀彻温大学）；University of Tulane（杜兰大学）
Breast cancer is one of the most common causes of cancer-related death in women worldwide. Early and accurate diagnosis of breast cancer may significantly increase the survival rate of patients. In this study, we aim to develop a fully automatic, deep learning-based, method using descriptor features extracted by Deep Convolutional Neural Network (DCNN) models and pooling operation for the classification of hematoxylin and eosin stain (H&E) histological breast cancer images provided as a part of the International Conference on Image Analysis and Recognition (ICIAR) 2018 Grand Challenge on BreAs…
Weather nowcasting consists of predicting meteorological components in the short term at high spatial resolutions. Due to its influence in many human activities, accurate nowcasting has recently gained plenty of attention. In this paper, we treat the nowcasting problem as an image-to-image translation problem using satellite imagery. We introduce Broad-UNet, a novel architecture based on the core UNet model, to efficiently address this problem. In particular, the proposed Broad-UNet is equipped with asymmetric parallel convolutions as well as Atrous Spatial Pyramid Pooling (ASPP) module. In th…
Accurate traffic prediction is a key ingredient to enable traffic management like rerouting cars to reduce road congestion or regulating traffic via dynamic speed limits to maintain a steady flow. A way to represent traffic data is in the form of temporally changing heatmaps visualizing attributes of traffic, such as speed and volume. In recent works, U-Net models have shown SOTA performance on traffic forecasting from heatmaps. We propose to combine the U-Net architecture with graph layers which improves spatial generalization to unseen road networks compared to a Vanilla U-Net. In particular…
Johns Hopkins University（约翰斯·霍普金斯大学）；Capacloud AI；University of Michigan（密歇根大学）
Early identification and prevention of various plant diseases in commercial farms and orchards is a key feature of precision agriculture technology. This paper presents a high-performance real-time fine-grain object detection framework that addresses several obstacles in plant disease detection that hinder the performance of traditional methods, such as, dense distribution, irregular morphology, multi-scale object classes, textural similarity, etc. The proposed model is built on an improved version of the You Only Look Once (YOLOv4) algorithm. The modified network architecture maximizes both d…
Block-Fused Attention-Driven Adaptively-Pooled ResNet Model for Improved Cervical Cancer Classification
Math of Data Science & Simulation (MODSS) Lab, Computer Science & Engineering（数据科学与模拟（MODSS）实验室，计算机科学与工程）；Indian Institute of Technology Indore（印度理工学院印度尔分校）；Computer Science & Engineering（计算机科学与工程）；Indian Institute of Information Technology Nagpur（印度信息学院纳格浦分校）
