Unsupervised embedding of trajectories captures the latent structure of scientific migration
Luddy School of Informatics, Computing, and Engineering, Indiana University（印第安纳大学勒迪信息、计算与工程学院）；Network Science Institute at Northeastern University（东北大学网络科学研究所）；Kellogg School of Management at Northwestern University（西北大学凯洛格管理学院）；Northwestern Institute on Complex Systems（西北大学复杂系统研究所）；Centre for Science and Technology Studies, Leiden University（莱顿大学科学技术研究中心）；Stellenbosch University（斯坦陵布什大学）；Pohang University of Science and Technology（浦项科技大学）
Human migration and mobility drives major societal phenomena including epidemics, economies, innovation, and the diffusion of ideas. Although human mobility and migration have been heavily constrained by geographic distance throughout the history, advances and globalization are making other factors such as language and culture increasingly more important. Advances in neural embedding models, originally designed for natural language, provide an opportunity to tame this complexity and open new avenues for the study of migration. Here, we demonstrate the ability of the model word2vec to encode nuanced relationships between discrete locations from migration trajectories, producing an accurate, dense, continuous, and meaningful vector-space representation. The resulting representation provides a functional distance between locations, as well as a digital double that can be distributed, re-used, and itself interrogated to understand the many dimensions of migration. We show that the unique power of word2vec to encode migration patterns stems from its mathematical equivalence with the gravity model of mobility. Focusing on the case of scientific migration, we apply word2vec to a database of three million migration trajectories of scientists derived from the affiliations listed on their publication records. Using techniques that leverage its semantic structure, we demonstrate that embeddings can learn the rich structure that underpins scientific migration, such as cultural, linguistic, and prestige relationships at multiple levels of granularity. Our results provide a theoretical foundation and methodological framework for using neural embeddings to represent and understand migration both within and beyond science.
Beijing University of Posts and Telecommunications（北京邮电大学）
Learning knowledge representation of scientific paper data is a problem to be solved, and how to learn the representation of paper nodes in scientific paper heterogeneous network is the core to solve this problem. This paper proposes an unsupervised cluster-level scientific paper heterogeneous graph node representation learning method (UCHL), aiming at obtaining the representation of nodes (authors, institutions, papers, etc.) in the heterogeneous graph of scientific papers. Based on the heterogeneous graph representation, this paper performs link prediction on the entire heterogeneous graph a…
Unsupervised Machine Learning for Scientific Discovery: Workflow and Best Practices
Department of Neuroscience, Baylor College of Medicine（神经科学系，贝勒医学院）；Department of Applied and Computational Mathematics and Statistics, University of Notre Dame（应用与计算数学和统计学系，圣母大学）；Department of Statistics, Columbia University（统计学系，哥伦比亚大学）；Zuckerman Institute & Irving Institute, Columbia University（Zuckerman研究所与Irving研究所，哥伦比亚大学）
Sandia National Laboratories（桑迪亚国家实验室）；Arizona State University（亚利桑那州立大学）
We introduce physics-informed multimodal autoencoders (PIMA) - a variational inference framework for discovering shared information in multimodal scientific datasets representative of high-throughput testing. Individual modalities are embedded into a shared latent space and fused through a product of experts formulation, enabling a Gaussian mixture prior to identify shared features. Sampling from clusters allows cross-modal generative modeling, with a mixture of expert decoder imposing inductive biases encoding prior scientific knowledge and imparting structured disentanglement of the latent s…
Graduate School of China Academy of Engineering Physics（中国工程物理研究院研究生院）
In recent years, machine learning methods have been used to assist scientists in scientific research. Human scientific theories are based on a series of concepts. How machine learns the concepts from experimental data will be an important first step. We propose a hybrid method to extract interpretable physical concepts through unsupervised machine learning. This method consists of two stages. At first, we need to find the Betti numbers of experimental data. Secondly, given the Betti numbers, we use a variational autoencoder network to extract meaningful physical variables. We test our protocol…
Recent advances in natural language processing have enabled automation of a wide range of tasks, including machine translation, named entity recognition, and sentiment analysis. Automated summarization of documents, or groups of documents, however, has remained elusive, with many efforts limited to extraction of keywords, key phrases, or key sentences. Accurate abstractive summarization has yet to be achieved due to the inherent difficulty of the problem, and limited availability of training data. In this paper, we propose a topic-centric unsupervised multi-document summarization framework to …
The scientific literature is a rich source of information for data mining with conceptual knowledge graphs; the open science movement has enriched this literature with complementary source code that implements scientific models. To exploit this new resource, we construct a knowledge graph using unsupervised learning methods to identify conceptual entities. We associate source code entities to these natural language concepts using word embedding and clustering techniques. Practical naming conventions for methods and functions tend to reflect the concept(s) they implement. We take advantage of t…
Carl von Ossietzky Universität Oldenburg（卡尔·冯·奥西茨基奥尔登堡大学）
In many scientific disciplines structures in high-dimensional data have to be found, e.g., in stellar spectra, in genome data, or in face recognition tasks. In this work we present a novel approach to non-linear dimensionality reduction. It is based on fitting K-nearest neighbor regression to the unsupervised regression framework for learning of low-dimensional manifolds. Similar to related approaches that are mostly based on kernel methods, unsupervised K-nearest neighbor (UNN) regression optimizes latent variables w.r.t. the data space reconstruction error employing the K-nearest neighbor he…
University of California at Davis（加利福尼亚大学戴维斯分校）；Intel Corporation（英特尔公司）；Lawrence Berkeley National Laboratory（劳伦斯伯克利国家实验室）；Baidu Research（百度研究院）
Extracting actionable insight from complex unlabeled scientific data is an open challenge and key to unlocking data-driven discovery in science. Complementary and alternative to supervised machine learning approaches, unsupervised physics-based methods based on behavior-driven theories hold great promise. Due to computational limitations, practical application on real-world domain science problems has lagged far behind theoretical development. We present our first step towards bridging this divide - DisCo - a high-performance distributed workflow for the behavior-driven local causal state theo…
Yale University（耶鲁大学）；Bar-Ilan University（巴伊兰大学）；Hebrew University of Jerusalem（希伯来大学）
Multi-modal high throughput biological data presents a great scientific opportunity and a significant computational challenge. In multi-modal measurements, every sample is observed simultaneously by two or more sets of sensors. In such settings, many observed variables in both modalities are often nuisance and do not carry information about the phenomenon of interest. Here, we propose a multi-modal unsupervised feature selection framework: identifying informative variables based on coupled high-dimensional measurements. Our method is designed to identify features associated with two types of l…
Institute of Formal and Applied Linguistics, Charles University（查尔斯大学形式与应用语言学研究所）
Authors' keyphrases assigned to scientific articles are essential for recognizing content and topic aspects. Most of the proposed supervised and unsupervised methods for keyphrase generation are unable to produce terms that are valuable but do not appear in the text. In this paper, we explore the possibility of considering the keyphrase string as an abstractive summary of the title and the abstract. First, we collect, process and release a large dataset of scientific paper metadata that contains 2.2 million records. Then we experiment with popular text summarization neural architectures. Despi…
Sapienza University of Rome（罗马第一大学）；ETH Zurich（苏黎世联邦理工学院）；Sony Computer Science Laboratories（索尼计算机科学实验室）；Complexity Science Hub（复杂性科学中心）
The perception of facial beauty is a complex phenomenon depending on many, detailed and global facial features influencing each other. In the machine learning community this problem is typically tackled as a problem of supervised inference. However, it has been conjectured that this approach does not capture the complexity of the phenomenon. A recent original experiment (Ibáñez-Berganza et al., Scientific Reports 9, 8364, 2019) allowed different human subjects to navigate the face-space and ``sculpt'' their preferred modification of a reference facial portrait. Here we present an unsupervised …
Medical Profile Model: Scientific and Practical Applications in Healthcare
National Institute of Standards and Technology（美国国家标准与技术研究院）；Rice University（莱斯大学）；Colorado State University（科罗拉多州立大学）；University of Colorado（科罗拉多大学）；National Research Institute for Metals（国家金属材料研究所）
University of Amsterdam（阿姆斯特丹大学）；York University（约克大学）；University of Toronto（多伦多大学）
Scientific imaging techniques such as optical and electron microscopy and computed tomography (CT) scanning are used to study the 3D structure of an object through 2D observations. These observations are related to the original 3D object through orthogonal integral projections. For common 3D reconstruction algorithms, computational efficiency requires the modeling of the 3D structures to take place in Fourier space by applying the Fourier slice theorem. At present, it is unclear how to differentiate through the projection operator, and hence current learning algorithms can not rely on gradient…
The clustering of data into physically meaningful subsets often requires assumptions regarding the number, size, or shape of the subgroups. Here, we present a new method, simultaneous coherent structure coloring (sCSC), which accomplishes the task of unsupervised clustering without a priori guidance regarding the underlying structure of the data. sCSC performs a sequence of binary splittings on the dataset such that the most dissimilar data points are required to be in separate clusters. To achieve this, we obtain a set of orthogonal coordinates along which dissimilarity in the dataset is maxi…
