Electron, optical, and scanning probe microscopy methods are generating ever increasing volume of image data containing information on atomic and mesoscale structures and functionalities. This necessitates the development of the machine learning methods for discovery of physical and chemical phenomena from the data, such as manifestations of symmetry breaking in electron and scanning tunneling microscopy images, variability of the nanoparticles. Variational autoencoders (VAEs) are emerging as a powerful paradigm for the unsupervised data analysis, allowing to disentangle the factors of variability and discover optimal parsimonious representation. Here, we summarize recent developments in VAEs, covering the basic principles and intuition behind the VAEs. The invariant VAEs are introduced as an approach to accommodate scale and translation invariances present in imaging data and separate known factors of variations from the ones to be discovered. We further describe the opportunities enabled by the control over VAE architecture, including conditional, semi-supervised, and joint VAEs. Several case studies of VAE applications for toy models and experimental data sets in Scanning Transmission Electron Microscopy are discussed, emphasizing the deep connection between VAE and basic physical principles. All the codes used here are available at https://github.com/saimani5/VAE-tutorials and this article can be used as an application guide when applying these to own data sets.
University of California Santa Barbara (UCSB)（加州大学圣塔芭芭拉分校）
We develop data-driven methods for incorporating physical information for priors to learn parsimonious representations of nonlinear systems arising from parameterized PDEs and mechanics. Our approach is based on Variational Autoencoders (VAEs) for learning from observations nonlinear state space models. We develop ways to incorporate geometric and topological priors through general manifold latent space representations. We investigate the performance of our methods for learning low dimensional representations for the nonlinear Burgers equation and constrained mechanical systems.
Reward driven discovery of the optimal microstructure representations with invariant variational autoencoders
Institute for Materials Science and Center for Nanointegration Duisburg-Essen (CENIDE), University of Duisburg-Essen（杜伊斯堡-埃森大学材料科学研究所与杜伊斯堡-埃森纳米集成中心）；Department of Materials Science and Engineering, University of Tennessee（田纳西大学材料科学与工程系）；Department of Material Science and Engineering, Tokyo Institute of Technology（东京工业大学材料科学与工程系）；Pacific Northwest National Laboratory（太平洋西北国家实验室）
KTH Royal Institute of Technology（瑞典皇家理工学院）；Universidad Politécnica de Madrid（马德里理工大学）；Universitat Politècnica de València（瓦伦西亚理工大学）；Swedish e-Science Research Centre (SeRC)（瑞典电子科学研究中心）
We propose a deep probabilistic-neural-network architecture for learning a minimal and near-orthogonal set of non-linear modes from high-fidelity turbulent-flow-field data useful for flow analysis, reduced-order modeling, and flow control. Our approach is based on $β$-variational autoencoders ($β$-VAEs) and convolutional neural networks (CNNs), which allow us to extract non-linear modes from multi-scale turbulent flows while encouraging the learning of independent latent variables and penalizing the size of the latent vector. Moreover, we introduce an algorithm for ordering VAE-based modes wit…
Molecular Representation Learning is essential to solving many drug discovery and computational chemistry problems. It is a challenging problem due to the complex structure of molecules and the vast chemical space. Graph representations of molecules are more expressive than traditional representations, such as molecular fingerprints. Therefore, they can improve the performance of machine learning models. We propose SeMole, a method that augments the Junction Tree Variational Autoencoders, a state-of-the-art generative model for molecular graphs, with semi-supervised learning. SeMole aims to im…
Adversarial Disentanglement by Backpropagation with Physics-Informed Variational Autoencoder
Department of Civil Engineering and Geosciences（土木工程与地球科学系）；Technical University of Delft（代尔夫特理工大学）；Department of Engineering（工程系）；University of Cambridge（剑桥大学）
Generative artificial intelligence for computational chemistry: a roadmap to predicting emergent phenomena
University of Maryland（马里兰大学）；Institute for Physical Science and Technology（物理科学与技术研究所）；University of Maryland Institute for Health Computing（马里兰大学健康计算研究所）；Department of Chemistry and Biochemistry（化学与生物化学系）；Department of Physics（物理系）；Biophysics Program（生物物理学项目）
UT-Battelle, LLC（UT-巴特尔有限责任公司）；U.S. Department of Energy（美国能源部）
Unsupervised and semi-supervised ML methods such as variational autoencoders (VAE) have become widely adopted across multiple areas of physics, chemistry, and materials sciences due to their capability in disentangling representations and ability to find latent manifolds for classification and regression of complex experimental data. Like other ML problems, VAEs require hyperparameter tuning, e.g., balancing the Kullback Leibler (KL) and reconstruction terms. However, the training process and resulting manifold topology and connectivity depend not only on hyperparameters, but also their evolut…
Sparse coding strategies have been lauded for their parsimonious representations of data that leverage low dimensional structure. However, inference of these codes typically relies on an optimization procedure with poor computational scaling in high-dimensional problems. For example, sparse inference in the representations learned in the high-dimensional intermediary layers of deep neural networks (DNNs) requires an iterative minimization to be performed at each training step. As such, recent, quick methods in variational inference have been proposed to infer sparse codes by learning a distrib…
MIRRA: Fast, High-Fidelity Molecular Generation with Chemistry-Informed Discrete Flow Matching
Courant Institute of Mathematical Sciences, New York University（纽约大学柯朗数学科学研究所）；Tandon School of Engineering, New York University（纽约大学坦登工程学院）；Department of Physics, New York University（纽约大学物理系）
Optimizing molecular design and discovering novel chemical structures to meet certain objectives, such as quantitative estimates of the drug-likeness score (QEDs), is NP-hard due to the vast combinatorial design space of discrete molecular structures, which makes it near impossible to explore the entire search space comprehensively to exploit de novo structures with properties of interest. To address this challenge, reducing the intractable search space into a lower-dimensional latent volume helps examine molecular candidates more feasibly via inverse design. Autoencoders are suitable deep lea…
J.P. Morgan（摩根大通）；The Alan Turing Institute（艾伦·图灵研究所）；University of Oxford（牛津大学）
Neural network based data-driven market simulation unveils a new and flexible way of modelling financial time series without imposing assumptions on the underlying stochastic dynamics. Though in this sense generative market simulation is model-free, the concrete modelling choices are nevertheless decisive for the features of the simulated paths. We give a brief overview of currently used generative modelling approaches and performance evaluation metrics for financial time series, and address some of the challenges to achieve good results in the latter. We also contrast some classical approache…
Indraprastha Institute of Information Technology（因德拉普拉斯信息技术学院）
In this work we address the problem of real-time dynamic medical MRI and X Ray CT image reconstruction from parsimonious samples Fourier frequency space for MRI and sinogram tomographic projections for CT. Today the de facto standard for such reconstruction is compressed sensing. CS produces high quality images (with minimal perceptual loss, but such reconstructions are time consuming, requiring solving a complex optimization problem. In this work we propose to learn the reconstruction from training samples using an autoencoder. Our work is based on the universal function approximation capacit…
University of Twente（特文特大学）；University of Washington（华盛顿大学）
Complex systems manifest a small number of instabilities and bifurcations that are canonical in nature, resulting in universal pattern forming characteristics as a function of some parametric dependence. Such parametric instabilities are mathematically characterized by their universal un-foldings, or normal form dynamics, whereby a parsimonious model can be used to represent the dynamics. Although center manifold theory guarantees the existence of such low-dimensional normal forms, finding them has remained a long standing challenge. In this work, we introduce deep learning autoencoders to dis…
With the recent advances in machine learning for quantum chemistry, it is now possible to predict the chemical properties of compounds and to generate novel molecules. Existing generative models mostly use a string- or graph-based representation, but the precise three-dimensional coordinates of the atoms are usually not encoded. First attempts in this direction have been proposed, where autoregressive or GAN-based models generate atom coordinates. Those either lack a latent space in the autoregressive setting, such that a smooth exploration of the compound space is not possible, or cannot gene…
Invariant Discovery of Features Across Multiple Length Scales: Applications in Microscopy and Autonomous Materials Characterization
University of Tennessee（田纳西大学）；Tokyo Institute of Technology（东京工业大学）；Oak Ridge National Laboratory（橡树岭国家实验室）；Pacific Northwest National Laboratory（太平洋西北国家实验室）
A semi-agnostic ansatz with variable structure for quantum machine learning
Universitat Autonoma de Barcelona（巴塞罗那自治大学）；Los Alamos National Laboratory（洛斯阿拉莫斯国家实验室）；Institute for Quantum Computing, University of Waterloo（滑铁卢大学量子计算研究所）
