Learning knowledge representation of scientific paper data is a problem to be solved, and how to learn the representation of paper nodes in scientific paper heterogeneous network is the core to solve this problem. This paper proposes an unsupervised cluster-level scientific paper heterogeneous graph node representation learning method (UCHL), aiming at obtaining the representation of nodes (authors, institutions, papers, etc.) in the heterogeneous graph of scientific papers. Based on the heterogeneous graph representation, this paper performs link prediction on the entire heterogeneous graph and obtains the relationship between the edges of the nodes, that is, the relationship between papers and papers. Experiments results show that the proposed method achieves excellent performance on multiple evaluation metrics on real scientific paper datasets.
Advanced Analytics Institute, University of Technology Sydney（悉尼科技大学高级分析研究所）；Dongguan University of Technology（东莞理工学院）
Complex categorical data is often hierarchically coupled with heterogeneous relationships between attributes and attribute values and the couplings between objects. Such value-to-object couplings are heterogeneous with complementary and inconsistent interactions and distributions. Limited research exists on unlabeled categorical data representations, ignores the heterogeneous and hierarchical couplings, underestimates data characteristics and complexities, and overuses redundant information, etc. The deep representation learning of unlabeled categorical data is challenging, overseeing such val…
École polytechnique fédérale de Lausanne (EPFL)（洛桑联邦理工学院）
In the era of misinformation and information inflation, the credibility assessment of the produced news is of the essence. However, fact-checking can be challenging considering the limited references presented in the news. This challenge can be transcended by utilizing the knowledge graph that is related to the news articles. In this work, we present a methodology for creating scientific news article representations by modeling the directed graph between the scientific news articles and the cited scientific publications. The network used for the experiments is comprised of the scientific news …
Walmart Labs（沃尔玛实验室）；Gartner（高德纳）；University of Arkansas（阿肯色大学）
Representation Learning in a heterogeneous space with mixed variables of numerical and categorical types has interesting challenges due to its complex feature manifold. Moreover, feature learning in an unsupervised setup, without class labels and a suitable learning loss function, adds to the problem complexity. Further, the learned representation and subsequent predictions should not reflect discriminatory behavior towards certain sensitive groups or attributes. The proposed feature map should preserve maximum variations present in the data and needs to be fair with respect to the sensitive v…
University of California at Davis（加利福尼亚大学戴维斯分校）；Intel Corporation（英特尔公司）；Lawrence Berkeley National Laboratory（劳伦斯伯克利国家实验室）；Baidu Research（百度研究院）
Extracting actionable insight from complex unlabeled scientific data is an open challenge and key to unlocking data-driven discovery in science. Complementary and alternative to supervised machine learning approaches, unsupervised physics-based methods based on behavior-driven theories hold great promise. Due to computational limitations, practical application on real-world domain science problems has lagged far behind theoretical development. We present our first step towards bridging this divide - DisCo - a high-performance distributed workflow for the behavior-driven local causal state theo…
Tsinghua University（清华大学）；AHI Fintech（安恒信息金融科技）；Youku Cognitive and Intelligent Lab, Alibaba Group（阿里巴巴集团优酷认知与智能实验室）
Since the label collecting is prohibitive and time-consuming, unsupervised methods are preferred in applications such as fraud detection. Meanwhile, such applications usually require modeling the intrinsic clusters in high-dimensional data, which usually displays heterogeneous statistical patterns as the patterns of different clusters may appear in different dimensions. Existing methods propose to model the data clusters on selected dimensions, yet globally omitting any dimension may damage the pattern of certain clusters. To address the above issues, we propose a novel unsupervised generative…
Microsoft Research Asia（微软亚洲研究院）；Peking University（北京大学）；University of Michigan（密歇根大学）
Unsupervised text embedding methods, such as Skip-gram and Paragraph Vector, have been attracting increasing attention due to their simplicity, scalability, and effectiveness. However, comparing to sophisticated deep learning architectures such as convolutional neural networks, these methods usually yield inferior results when applied to particular machine learning tasks. One possible reason is that these text embedding methods learn the representation of text in a fully unsupervised way, without leveraging the labeled information available for the task. Although the low dimensional representa…
Unsupervised Machine Learning for Scientific Discovery: Workflow and Best Practices
Department of Neuroscience, Baylor College of Medicine（神经科学系，贝勒医学院）；Department of Applied and Computational Mathematics and Statistics, University of Notre Dame（应用与计算数学和统计学系，圣母大学）；Department of Statistics, Columbia University（统计学系，哥伦比亚大学）；Zuckerman Institute & Irving Institute, Columbia University（Zuckerman研究所与Irving研究所，哥伦比亚大学）
An Efficient Unsupervised Federated Learning Approach for Anomaly Detection in Heterogeneous IoT Networks
Faculty of Electrical and Computer Engineering, Qom University of Technology（电气与计算机工程学院，科姆技术大学）；Department of Computer Engineering, Faculty of Industry and Mining (Khash), Sistan and Baluchestan University（计算机工程系，工业与采矿学院（Khash），锡斯坦和巴尔赫斯坦大学）；Canadian Institute for Cybersecurity, University of New Brunswick（加拿大网络安全研究所，新不伦瑞克大学）；Sustainable Digitalisation Research Centre (SDRC), Department of Computer Science and Media Technology, Faculty of Technology and Society, Malmo University（可持续数字化研究中心（SDRC），计算机科学与媒体技术系，技术与社会学院，马尔默大学）
Sandia National Laboratories（桑迪亚国家实验室）；Arizona State University（亚利桑那州立大学）
We introduce physics-informed multimodal autoencoders (PIMA) - a variational inference framework for discovering shared information in multimodal scientific datasets representative of high-throughput testing. Individual modalities are embedded into a shared latent space and fused through a product of experts formulation, enabling a Gaussian mixture prior to identify shared features. Sampling from clusters allows cross-modal generative modeling, with a mixture of expert decoder imposing inductive biases encoding prior scientific knowledge and imparting structured disentanglement of the latent s…
Adaptive Graph Mixture of Residual Experts: Unsupervised Learning on Diverse Graphs with Heterogeneous Specialization
School of New Media and Communication, Tianjin University（新媒体与传播学院，天津大学）；Baidu（百度）；School of Computer Science and Engineering, Beihang University（计算机科学与工程学院，北航大学）
We introduce a framework to leverage knowledge acquired from a repository of (heterogeneous) supervised datasets to new unsupervised datasets. Our perspective avoids the subjectivity inherent in unsupervised learning by reducing it to supervised learning, and provides a principled way to evaluate unsupervised algorithms. We demonstrate the versatility of our framework via simple agnostic bounds on unsupervised problems. In the context of clustering, our approach helps choose the number of clusters and the clustering algorithm, remove the outliers, and provably circumvent the Kleinberg's imposs…
Domain adaptation leverages the knowledge in one domain - the source domain - to improve learning efficiency in another domain - the target domain. Existing heterogeneous domain adaptation research is relatively well-progressed, but only in situations where the target domain contains at least a few labeled instances. In contrast, heterogeneous domain adaptation with an unlabeled target domain has not been well-studied. To contribute to the research in this emerging field, this paper presents: (1) an unsupervised knowledge transfer theorem that guarantees the correctness of transferring knowled…
Quadratic Neuron-empowered Heterogeneous Autoencoder for Unsupervised Anomaly Detection
Harbin Institute of Technology（哈尔滨工业大学）；The Hong Kong Polytechnic University（香港理工大学）；Weill Cornell Medicine（威尔康奈尔医学院）；Cornell University（康奈尔大学）；The Chinese University of Hong Kong（香港中文大学）
Technical University of Munich（慕尼黑工业大学）；Mercateo AG（Mercateo公司）
Heterogeneous Information Network (HIN) embedding refers to the low-dimensional projections of the HIN nodes that preserve the HIN structure and semantics. HIN embedding has emerged as a promising research field for network analysis as it enables downstream tasks such as clustering and node classification. In this work, we propose \ours for joint learning of cluster embeddings as well as cluster-aware HIN embedding. We assume that the connected nodes are highly likely to fall in the same cluster, and adopt a variational approach to preserve the information in the pairwise relations in a cluste…
Conventional mutual information (MI) based feature selection (FS) methods are unable to handle heterogeneous feature subset selection properly because of data format differences or estimation methods of MI between feature subset and class label. A way to solve this problem is feature transformation (FT). In this study, a novel unsupervised feature transformation (UFT) which can transform non-numerical features into numerical features is developed and tested. The UFT process is MI-based and independent of class label. MI-based FS algorithms, such as Parzen window feature selector (PWFS), minimu…
Graduate School of China Academy of Engineering Physics（中国工程物理研究院研究生院）
In recent years, machine learning methods have been used to assist scientists in scientific research. Human scientific theories are based on a series of concepts. How machine learns the concepts from experimental data will be an important first step. We propose a hybrid method to extract interpretable physical concepts through unsupervised machine learning. This method consists of two stages. At first, we need to find the Betti numbers of experimental data. Secondly, given the Betti numbers, we use a variational autoencoder network to extract meaningful physical variables. We test our protocol…
Carnegie Mellon University（卡内基梅隆大学）；Peking University（北京大学）；Purdue University（普渡大学）；Arima Inc.（Arima公司）；IQVIA（艾昆纬）；University of Illinois at Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）
Outlier detection (OD) is a key machine learning (ML) task for identifying abnormal objects from general samples with numerous high-stake applications including fraud detection and intrusion detection. Due to the lack of ground truth labels, practitioners often have to build a large number of unsupervised, heterogeneous models (i.e., different algorithms with varying hyperparameters) for further combination and analysis, rather than relying on a single model. How to accelerate the training and scoring on new-coming samples by outlyingness (referred as prediction throughout the paper) with a la…
University of Electronic Science and Technology of China（电子科技大学）；Iowa State University（爱荷华州立大学）；University of Maryland（马里兰大学）
Quantifying and predicting the long-term impact of scientific writings or individual scholars has important implications for many policy decisions, such as funding proposal evaluation and identifying emerging research fields. In this work, we propose an approach based on Heterogeneous Dynamical Graph Neural Network (HDGNN) to explicitly model and predict the cumulative impact of papers and authors. HDGNN extends heterogeneous GNNs by incorporating temporally evolving characteristics and capturing both structural properties of attributed graph and the growing sequence of citation behavior. HDGN…
UCL（伦敦大学学院）；UCLH NHS Foundation Trust（伦敦大学学院医院国民保健服务基金会信托）
Any novel medical imaging modality that differs from previous protocols e.g. in the number of imaging channels, introduces a new domain that is heterogeneous from previous ones. This common medical imaging scenario is rarely considered in the domain adaptation literature, which handles shifts across domains of the same dimensionality. In our work we rely on stochastic generative modeling to translate across two heterogeneous domains at pixel space and introduce two new loss functions that promote semantic consistency. Firstly, we introduce a semantic cycle-consistency loss in the source domain…
Recent advances in natural language processing have enabled automation of a wide range of tasks, including machine translation, named entity recognition, and sentiment analysis. Automated summarization of documents, or groups of documents, however, has remained elusive, with many efforts limited to extraction of keywords, key phrases, or key sentences. Accurate abstractive summarization has yet to be achieved due to the inherent difficulty of the problem, and limited availability of training data. In this paper, we propose a topic-centric unsupervised multi-document summarization framework to …
MIARec: Mutual-influence-aware Heterogeneous Network Embedding for Scientific Paper Recommendation
College of Computer and Information Science, Southwest university（西南大学计算机与信息科学学院）；College of Computer and Information Science, Chongqing normal university（重庆师范大学计算机与信息科学学院）
UiT The Arctic University of Norway（挪威北极大学）；University of Genoa（热那亚大学）
Change detection in heterogeneous multitemporal satellite images is an emerging and challenging topic in remote sensing. In particular, one of the main challenges is to tackle the problem in an unsupervised manner. In this paper we propose an unsupervised framework for bitemporal heterogeneous change detection based on the comparison of affinity matrices and image regression. First, our method quantifies the similarity of affinity matrices computed from co-located image patches in the two images. This is done to automatically identify pixels that are likely to be unchanged. With the identified…
A foundation model for energy and radiation systems built on heterogeneous scientific interfaces
University of Illinois Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）；Indian Institute of Technology Delhi（印度理工学院德里分校）；Yardi School of Artificial Intelligence (ScAI)（雅迪人工智能学院）；National Center for Supercomputing Applications (NCSA)（国家超级计算应用中心）
The scientific literature is a rich source of information for data mining with conceptual knowledge graphs; the open science movement has enriched this literature with complementary source code that implements scientific models. To exploit this new resource, we construct a knowledge graph using unsupervised learning methods to identify conceptual entities. We associate source code entities to these natural language concepts using word embedding and clustering techniques. Practical naming conventions for methods and functions tend to reflect the concept(s) they implement. We take advantage of t…
Purdue University（普渡大学）；Ahsanullah University of Science and Technology（阿萨努拉科技大学）
Anomaly detection (AD) in a surveillance scenario is an emerging and challenging field of research. For autonomous vehicles like drones or cars, it is immensely important to distinguish between normal and abnormal states in real-time. Additionally, we also need to detect any device malfunction. But the nature and degree of abnormality may vary depending upon the actual environment and adversary. As a result, it is impractical to model all cases a-priori and use supervised methods to classify. Also, an autonomous vehicle provides various data types like images and other analog or digital sensor…
Learning unsupervised node embeddings facilitates several downstream tasks such as node classification and link prediction. A node embedding is universal if it is designed to be used by and benefit various downstream tasks. This work introduces PanRep, a graph neural network (GNN) model, for unsupervised learning of universal node representations for heterogenous graphs. PanRep consists of a GNN encoder that obtains node embeddings and four decoders, each capturing different topological and node feature properties. Abiding to these properties the novel unsupervised framework learns universal e…
MAPSeg: Unified Unsupervised Domain Adaptation for Heterogeneous Medical Image Segmentation Based on 3D Masked Autoencoding and Pseudo-Labeling
Columbia University（哥伦比亚大学）；Télécom Paris（巴黎电信学院）；Institut Polytechnique de Paris（巴黎综合理工学院）；University of Maryland, College Park（马里兰大学帕克分校）；University of California, Irvine（加州大学欧文分校）；University of Rochester（罗切斯特大学）；Universidade Federal de São Paulo（圣保罗联邦大学）；Duke University（杜克大学）；Emory University（埃默里大学）
Lawrence Livermore National Laboratory（劳伦斯利弗莫尔国家实验室）；Kaiser Permanente Division of Research（凯撒永久医疗研究部）
We develop an unsupervised probabilistic model for heterogeneous Electronic Health Record (EHR) data. Utilizing a mixture model formulation, our approach directly models sequences of arbitrary length, such as medications and laboratory results. This allows for subgrouping and incorporation of the dynamics underlying heterogeneous data types. The model consists of a layered set of latent variables that encode underlying structure in the data. These variables represent subject subgroups at the top layer, and unobserved states for sequences in the second layer. We train this model on episodic dat…
University of Bath（巴斯大学）；University of Cambridge（剑桥大学）
Glioblastoma is profoundly heterogeneous in regional microstructure and vasculature. Characterizing the spatial heterogeneity of glioblastoma could lead to more precise treatment. With unsupervised learning techniques, glioblastoma MRI-derived radiomic features have been widely utilized for tumor sub-region segmentation and survival prediction. However, the reliability of algorithm outcomes is often challenged by both ambiguous intermediate process and instability introduced by the randomness of clustering algorithms, especially for data from heterogeneous patients. In this paper, we propose a…
Univ. Grenoble Alpes（格勒诺布尔阿尔卑斯大学）；ITMO University（圣彼得堡国立信息技术机械与光学大学）；University of Washington（华盛顿大学）
Since time immemorial, people have been looking for ways to organize scientific knowledge into some systems to facilitate search and discovery of new ideas. The problem was partially solved in the pre-Internet era using library classifications, but nowadays it is nearly impossible to classify all scientific and popular scientific knowledge manually. There is a clear gap between the diversity and the amount of data available on the Internet and the algorithms for automatic structuring of such data. In our preliminary study, we approach the problem of knowledge discovery on web-scale data with d…
Physics-informed and Unsupervised Riemannian Domain Adaptation for Machine Learning on Heterogeneous EEG Datasets
University Paris-Saclay（巴黎萨克雷大学）；Inria（法国国家信息与自动化研究所）；CEA（法国原子能和替代能源委员会）；TAU Inria（特拉维夫大学-法国国家信息与自动化研究所联合机构）；LISN-CNRS（法国国家科学研究中心LISN实验室）；Roche Pharma Research and Early Development（罗氏制药研发部）；Roche Innovation Center Basel（罗氏创新中心巴塞尔分部）；F. Hoffmann–La Roche Ltd（罗氏控股有限公司）
Unsupervised embedding of trajectories captures the latent structure of scientific migration
Luddy School of Informatics, Computing, and Engineering, Indiana University（印第安纳大学勒迪信息、计算与工程学院）；Network Science Institute at Northeastern University（东北大学网络科学研究所）；Kellogg School of Management at Northwestern University（西北大学凯洛格管理学院）；Northwestern Institute on Complex Systems（西北大学复杂系统研究所）；Centre for Science and Technology Studies, Leiden University（莱顿大学科学技术研究中心）；Stellenbosch University（斯坦陵布什大学）；Pohang University of Science and Technology（浦项科技大学）
KAIST（韩国科学技术院）；Institute for Basic Science（基础科学研究所）；Microsoft Research Asia（微软亚洲研究院）；Tsinghua University（清华大学）
This paper presents FedX, an unsupervised federated learning framework. Our model learns unbiased representation from decentralized and heterogeneous local data. It employs a two-sided knowledge distillation with contrastive learning as a core component, allowing the federated system to function without requiring clients to share any data features. Furthermore, its adaptable architecture can be used as an add-on module for existing unsupervised algorithms in federated settings. Experiments show that our model improves performance significantly (1.58--5.52pp) on five unsupervised algorithms.
Florida State University（佛罗里达州立大学）；JD Finance America Corporation（京东金融美国公司）
Graph representation learning is to learn universal node representations that preserve both node attributes and structural information. The derived node representations can be used to serve various downstream tasks, such as node classification and node clustering. When a graph is heterogeneous, the problem becomes more challenging than the homogeneous graph node learning problem. Inspired by the emerging information theoretic-based learning algorithm, in this paper we propose an unsupervised graph neural network Heterogeneous Deep Graph Infomax (HDGI) for heterogeneous graph representation lea…
Ocean University of China（中国海洋大学）；The University of Hong Kong（香港大学）；Shandong University of Science and Technology（山东科技大学）
Heterogeneous graph convolutional networks have gained great popularity in tackling various network analytical tasks on heterogeneous network data, ranging from link prediction to node classification. However, most existing works ignore the relation heterogeneity with multiplex network between multi-typed nodes and different importance of relations in meta-paths for node embedding, which can hardly capture the heterogeneous structure signals across different relations. To tackle this challenge, this work proposes a Multiplex Heterogeneous Graph Convolutional Network (MHGCN) for heterogeneous n…
Xi’an Jiaotong University（西安交通大学）；University of Virginia（弗吉尼亚大学）；University of Washington（华盛顿大学）；Ant Financial（蚂蚁金服）
Graph anomaly detection on attributed networks has become a prevalent research topic due to its broad applications in many influential domains. In real-world scenarios, nodes and edges in attributed networks usually display distinct heterogeneity, i.e. attributes of different types of nodes show great variety, different types of relations represent diverse meanings. Anomalies usually perform differently from the majority in various perspectives of heterogeneity in these networks. However, existing graph anomaly detection approaches do not leverage heterogeneity in attributed networks, which is…
