Deep learning based latent representations have been widely used for numerous scientific visualization applications such as isosurface similarity analysis, volume rendering, flow field synthesis, and data reduction, just to name a few. However, existing latent representations are mostly generated from raw data in an unsupervised manner, which makes it difficult to incorporate domain interest to control the size of the latent representations and the quality of the reconstructed data. In this paper, we present a novel importance-driven latent representation to facilitate domain-interest-guided scientific data visualization and analysis. We utilize spatial importance maps to represent various scientific interests and take them as the input to a feature transformation network to guide latent generation. We further reduced the latent size by a lossless entropy encoding algorithm trained together with the autoencoder, improving the storage and memory efficiency. We qualitatively and quantitatively evaluate the effectiveness and efficiency of latent representations generated by our method with data from multiple scientific visualization applications.
Feature related particle data analysis plays an important role in many scientific applications such as fluid simulations, cosmology simulations and molecular dynamics. Compared to conventional methods that use hand-crafted feature descriptors, some recent studies focus on transforming the data into a new latent space, where features are easier to be identified, compared and extracted. However, it is challenging to transform particle data into latent representations, since the convolution neural networks used in prior studies require the data presented in regular grids. In this paper, we adopt …
Scientifically-Interpretable Reasoning Network (ScIReN): Discovering Hidden Relationships in the Carbon Cycle and Beyond
Department of Informatics and Intelligent Systems, Institute of Energy and the Environment, The Pennsylvania State University（信息学与智能系统系，能源与环境研究所，宾夕法尼亚州立大学）
Sandia National Laboratories（桑迪亚国家实验室）；Arizona State University（亚利桑那州立大学）
We introduce physics-informed multimodal autoencoders (PIMA) - a variational inference framework for discovering shared information in multimodal scientific datasets representative of high-throughput testing. Individual modalities are embedded into a shared latent space and fused through a product of experts formulation, enabling a Gaussian mixture prior to identify shared features. Sampling from clusters allows cross-modal generative modeling, with a mixture of expert decoder imposing inductive biases encoding prior scientific knowledge and imparting structured disentanglement of the latent s…
Sichuan University（四川大学）；Huazhong University of Technology（华中理工大学）；New York University（纽约大学）
Generative Adversarial Networks (GAN) training process, in most cases, apply Uniform or Gaussian sampling methods in the latent space, which probably spends most of the computation on examples that can be properly handled and easy to generate. Theoretically, importance sampling speeds up stochastic optimization in supervised learning by prioritizing training examples. In this paper, we explore the possibility of adapting importance sampling into adversarial learning. We use importance sampling to replace Uniform and Gaussian sampling methods in the latent space and employ normalizing flow to a…
Latent-Space Importance Weighting under Covariate Shift
Wuhan University（武汉大学）；The Hong Kong University of Science and Technology（香港科技大学）；HKUST Shenzhen-Hong Kong Collaborative Innovation Research Institute（港深创新研究院）
Beijing University of Posts and Telecommunications（北京邮电大学）；Duke University（杜克大学）；Stanford University（斯坦福大学）；Weill Cornell Medical College（威尔康奈尔医学院）
Probabilistic generative models are attractive for scientific modeling because their inferred parameters can be used to generate hypotheses and design experiments. This requires that the learned model provide an accurate representation of the input data and yield a latent space that effectively predicts outcomes relevant to the scientific question. Supervised Variational Autoencoders (SVAEs) have previously been used for this purpose, where a carefully designed decoder can be used as an interpretable generative model while the supervised objective ensures a predictive latent representation. Un…
Argonne National Laboratory（阿贡国家实验室）；Carnegie Mellon University（卡内基梅隆大学）
The goal of this work is to address two limitations in autoencoder-based models: latent space interpretability and compatibility with unstructured meshes. This is accomplished here with the development of a novel graph neural network (GNN) autoencoding architecture with demonstrations on complex fluid flow applications. To address the first goal of interpretability, the GNN autoencoder achieves reduction in the number nodes in the encoding stage through an adaptive graph reduction procedure. This reduction procedure essentially amounts to flowfield-conditioned node sampling and sensor identifi…
Affinity-VAE: incorporating prior knowledge in representation learning from scientific images
Alan Turing Institute（阿兰·图灵研究所）；Science and Technology Facilities Council（科学与技术设施委员会）；University of Oxford（牛津大学）；Rosalind Franklin Institute（罗莎琳德·富兰克林研究所）
Department of Mathematics, Emory University（埃默里大学数学系）；School of Mathematical and Statistical Sciences, Clemson University（克林斯敦大学数学与统计科学学院）；Argonne National Laboratory（阿贡国家实验室）
Unsupervised embedding of trajectories captures the latent structure of scientific migration
Luddy School of Informatics, Computing, and Engineering, Indiana University（印第安纳大学勒迪信息、计算与工程学院）；Network Science Institute at Northeastern University（东北大学网络科学研究所）；Kellogg School of Management at Northwestern University（西北大学凯洛格管理学院）；Northwestern Institute on Complex Systems（西北大学复杂系统研究所）；Centre for Science and Technology Studies, Leiden University（莱顿大学科学技术研究中心）；Stellenbosch University（斯坦陵布什大学）；Pohang University of Science and Technology（浦项科技大学）
University of Cambridge（剑桥大学）；Microsoft Research Cambridge（微软剑桥研究院）；Tehran Institute for Advanced Studies（德黑兰高等研究院）
Variational Autoencoders (VAEs) are known to suffer from learning uninformative latent representation of the input due to issues such as approximated posterior collapse, or entanglement of the latent space. We impose an explicit constraint on the Kullback-Leibler (KL) divergence term inside the VAE objective function. While the explicit constraint naturally avoids posterior collapse, we use it to further understand the significance of the KL term in controlling the information transmitted through the VAE channel. Within this framework, we explore different properties of the estimated posterior…
A Guide to Feature Importance Methods for Scientific Inference
LMU Munich（慕尼黑大学）；Munich Center for Machine Learning (MCML)（慕尼黑机器学习中心）；Leibniz Institute for Prevention Research & Epidemiology – BIPS（莱布尼茨预防研究与流行病学研究所（BIPS））；University of Bremen（不来梅大学）；University of Copenhagen（哥本哈根大学）；University of Tübingen（蒂宾根大学）；Tübingen AI Center（蒂宾根人工智能中心）
GraphComp: Extreme Error-bounded Compression of Scientific Data via Temporal Graph Autoencoders
Computer, Electrical and Mathematical Sciences and Engineering Division, KAUST（卡奥斯特大学计算机、电气与数学科学与工程分校）；Department of Informatics and Telecommunications, University of Peloponnese（希腊皮埃尼亚斯大学信息与电信系）；Physical Science and Engineering Division, KAUST（卡奥斯特大学物理科学与工程分校）
Janelia Research Campus（珍妮亚研究园区）；HHMI（霍华德·休斯医学研究所）
The Importance Weighted Auto Encoder (IWAE) objective has been shown to improve the training of generative models over the standard Variational Auto Encoder (VAE) objective. Here, we derive importance weighted extensions to AVB and AAE. These latent variable models use implicitly defined inference networks whose approximate posterior density q_ϕ(z|x) cannot be directly evaluated, an essential ingredient for importance weighting. We show improved training and inference in latent variable models with our adversarially trained importance weighting method, and derive new theoretical connections be…
LIPN（巴黎第十三大学计算机科学实验室）；Université Sorbonne Paris Nord（索邦巴黎北大学）；INRIA Paris（法国国家信息与自动化研究所巴黎分部）
Even though Variational Autoencoders (VAEs) are widely used for semi-supervised learning, the reason why they work remains unclear. In fact, the addition of the unsupervised objective is most often vaguely described as a regularization. The strength of this regularization is controlled by down-weighting the objective on the unlabeled part of the training set. Through an analysis of the objective of semi-supervised VAEs, we observe that they use the posterior of the learned generative model to guide the inference model in learning the partially observed latent variable. We show that given this …
Korea Advanced Institute of Science and Technology（韩国科学技术院）；Electronics and Telecommunications Research Institute（韩国电子通信研究院）
Recently, many neural network-based image compression methods have shown promising results superior to the existing tool-based conventional codecs. However, most of them are often trained as separate models for different target bit rates, thus increasing the model complexity. Therefore, several studies have been conducted for learned compression that supports variable rates with single models, but they require additional network modules, layers, or inputs that often lead to complexity overhead, or do not provide sufficient coding efficiency. In this paper, we firstly propose a selective compre…
Variational autoencoders are among the most popular methods for distilling low-dimensional structure from high-dimensional data, making them increasingly valuable as tools for data exploration and scientific discovery. However, unlike typical machine learning problems in which a single model is trained once on a single large dataset, scientific workflows privilege learned features that are reproducible, portable across labs, and capable of incrementally adding new data. Ideally, methods used by different research groups should produce comparable results, even without sharing fully trained mode…
MiST: Understanding the Role of Mid-Stage Scientific Training in Developing Chemical Reasoning Models
Laboratory of Artificial Chemical Intelligence (LIAC), Institute of Chemical Sciences and Engineering (ISIC)（人工化学智能实验室（LIAC）、化学科学与工程研究所）；National Centre of Competence in Research (NCCR) Catalysis（催化研究国家级竞争力中心）；School of Photovoltaic and Renewable Energy engineering（光伏与可再生能源工程学院）；School of Computer Science and Engineering（计算机科学与工程学院）；Shanghai Artificial Intelligence Laboratory（上海人工智能实验室）；Green Dynamics
Exploring Autoencoder-based Error-bounded Compression for Scientific Data
University of California, Riverside（加州大学河滨分校）；Argonne National Laboratory（阿贡国家实验室）；Washington State University（华盛顿州立大学）；Missouri University of Science and Technology（密苏里科技大学）；University of Illinois at Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）
Compositional data are non-negative data collected in a rectangular matrix with a constant row sum. Due to the non-negativity the focus is on conditional proportions that add up to 1 for each row. A row of conditional proportions is called an observed budget. Latent budget analysis (LBA) assumes a mixture of latent budgets that explains the observed budgets. LBA is usually fitted to a contingency table, where the rows are levels of one or more explanatory variables and the columns the levels of a response variable. In prospective studies, there is only knowledge about the explanatory variables…
Machine learning lies at the heart of new possibilities for scientific discovery, knowledge generation, and artificial intelligence. Its potential benefits to these fields requires going beyond predictive accuracy and focusing on interpretability. In particular, many scientific problems require interpretations in a domain-specific interpretable feature space (e.g. the frequency domain) whereas attributions to the raw features (e.g. the pixel space) may be unintelligible or even misleading. To address this challenge, we propose TRIM (TRansformation IMportance), a novel approach which attributes…
Effective Latent Differential Equation Models via Attention and Multiple Shooting
Universidad de Buenos Aires（布宜诺斯艾利斯大学）；Mila - Quebec AI Institute（米拉-魁北克人工智能研究所）；University of Montreal（蒙特利尔大学）；IBM Research（IBM研究院）；CONICET（国家科学技术研究委员会）；CHU Sainte-Justine（圣贾斯汀大学医院中心）
Department of Electronics, Information and Bioengineering, Politecnico di Milano（电子工程与信息学院，米兰理工学院）；Osservatorio Astronomico di Roma, INAF（罗马天文台，INAF）；AstroAI, Center for Astrophysics | | Harvard & Smithsonian（AstroAI，哈佛与史密松尼天体物理中心）；Center for Computational Astrophysics, Institute for Advanced Study/The Flatiron Institute（计算天文学中心，高级研究院/Flatiron研究所）
Variational decomposition autoencoding improves disentanglement of latent representations
Dept. of Biomedical Engineering, Khalifa University of Science and Technology（生物医学工程系，哈利法大学）；School of Electrical and Computer Engineering, Aristotle University of Thessaloniki（电气与计算机工程学院，亚里士多德大学）
Extracting biomedical relations from large corpora of scientific documents is a challenging natural language processing task. Existing approaches usually focus on identifying a relation either in a single sentence (mention-level) or across an entire corpus (pair-level). In both cases, recent methods have achieved strong results by learning a point estimate to represent the relation; this is then used as the input to a relation classifier. However, the relation expressed in text between a pair of biomedical entities is often more complex than can be captured by a point estimate. To address this…
Max Planck Institute for Intelligent Systems（马克斯·普朗克智能系统研究所）
Autoencoders exhibit impressive abilities to embed the data manifold into a low-dimensional latent space, making them a staple of representation learning methods. However, without explicit supervision, which is often unavailable, the representation is usually uninterpretable, making analysis and principled progress challenging. We propose a framework, called latent responses, which exploits the locally contractive behavior exhibited by variational autoencoders to explore the learned manifold. More specifically, we develop tools to probe the representation using interventions in the latent spac…
