Reinforced dynamics for enhanced sampling in large atomic and molecular systems
Princeton University（普林斯顿大学）；Institute of Applied Physics and Computational Mathematics（应用物理与计算数学研究所）；CAEP Software Center for High Performance Numerical Simulation（中国工程物理研究院高性能数值模拟软件中心）；Beijing Institute of Big Data Research（北京大数据研究院）
A new approach for efficiently exploring the configuration space and computing the free energy of large atomic and molecular systems is proposed, motivated by an analogy with reinforcement learning. There are two major components in this new approach. Like metadynamics, it allows for an efficient exploration of the configuration space by adding an adaptively computed biasing potential to the original dynamics. Like deep reinforcement learning, this biasing potential is trained on the fly using deep neural networks, with data collected judiciously from the exploration and an uncertainty indicator from the neural network model playing the role of the reward function. Parameterization using neural networks makes it feasible to handle cases with a large set of collective variables. This has the potential advantage that selecting precisely the right set of collective variables has now become less critical for capturing the structural transformations of the system. The method is illustrated by studying the full-atom, explicit solvent models of alanine dipeptide and tripeptide, as well as the system of a polyalanine-10 molecule with 20 collective variables.
We show that the equations of reinforcement learning and light transport simulation are related integral equations. Based on this correspondence, a scheme to learn importance while sampling path space is derived. The new approach is demonstrated in a consistent light transport simulation algorithm that uses reinforcement learning to progressively learn where light comes from. As using this information for importance sampling includes information about visibility, too, the number of light transport paths with zero contribution is dramatically reduced, resulting in much less noisy images within …
Exploratory data analytics (EDA) is a sequential decision making process where analysts choose subsequent queries that might lead to some interesting insights based on the previous queries and corresponding results. Data processing systems often execute the queries on samples to produce results with low latency. Different downsampling strategy preserves different statistics of the data and have different magnitude of latency reductions. The optimum choice of sampling strategy often depends on the particular context of the analysis flow and the hidden intent of the analyst. In this paper, we ar…
Most of todays work on knowledge graph completion is concerned with sub-symbolic approaches that focus on the concept of embedding a given graph in a low dimensional vector space. Against this trend, we propose an approach called AnyBURL that is rooted in the symbolic space. Its core algorithm is based on sampling paths, which are generalized into Horn rules. Previously published results show that the prediction quality of AnyBURL is on the same level as current state of the art with the additional benefit of offering an explanation for the predicted fact. In this paper, we are concerned with …
Enhanced sampling methods such as metadynamics and umbrella sampling have become essential tools for exploring the configuration space of molecules and materials. At the same time, they have long faced a number of issues such as the inefficiency when dealing with a large number of collective variables (CVs) or systems with high free energy barriers. In this work, we show that with \redc{the clustering and adaptive tuning techniques}, the reinforced dynamics (RiD) scheme can be used to efficiently explore the configuration space and free energy landscapes with a large number of CVs or systems w…
DISA: Offline Importance Sampling for Distribution-Matching LLM-RL
Shanghai Jiao Tong University（上海交通大学）；Qwen Team, Alibaba Group（阿里集团Qwen团队）；The Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州））；The University of Science and Technology of China（中国科学技术大学）；Nanjing University（南京大学）；The University of Hong Kong（香港大学）
ANEO；Centre Borelli - ENS Paris-saclay（巴黎萨克雷高等师范学院博雷利中心）；Criteo AI Lab（科韬人工智能实验室）；Crest, ENSAE（法国国立统计与经济管理学院 Crest 研究中心）
The objective of offline RL is to learn optimal policies when a fixed exploratory demonstrations data-set is available and sampling additional observations is impossible (typically if this operation is either costly or rises ethical questions). In order to solve this problem, off the shelf approaches require a properly defined cost function (or its evaluation on the provided data-set), which are seldom available in practice. To circumvent this issue, a reasonable alternative is to query an expert for few optimal demonstrations in addition to the exploratory data-set. The objective is then to l…
EASAL (Efficient Atlasing and Sampling of Assembly Landscapes) is a geometric method for sampling and computing integrals over the potential energy landscape of small molecular assemblies. EASAL's efficiency arises from the fact that small assembly landscapes permit the use of so-called Cayley (inter-atomic distance based) parameters for geometric representation and sampling of the assembly configuration space regions; this results in their isolation, convexification, customized sampling and systematic traversal using a comprehensive topological roadmap. We define custom-designed measurements …
Computing accurate reaction rates is a central challenge in computational chemistry and biology because of the high cost of free energy estimation with unbiased molecular dynamics. In this work, a data-driven machine learning algorithm is devised to learn collective variables with a multitask neural network, where a common upstream part reduces the high dimensionality of atomic configurations to a low dimensional latent space, and separate downstream parts map the latent space to predictions of basin class labels and potential energies. The resulting latent space is shown to be an effective lo…
Designing an appropriate set of collective variables is crucial to the success of several enhanced sampling methods. Here we focus on how to obtain such variables from information limited to the metastable states. We characterize these states by a large set of descriptors and employ neural networks to compress this information in a lower-dimensional space, using Fisher's linear discriminant as an objective function to maximize the discriminative power of the network. We test this method on alanine dipeptide, using the non-linearly separable dataset composed by atomic distances. We then study a…
Best of Both Worlds: Uniform sampling in Cartesian and Cayley Molecular Assembly Configuration Space
EASAL（efficient atlasing and sampling of assembly landscape）是一种用于表征、可视化、采样和计算小分子组装势能面积分的几何方法。EASAL的效率来自于这样一个事实，即小装配景观允许使用所谓的凯莱参数（原子间距离）的几何表示和采样的装配配置空间区域;这导致他们的隔离，凸化，定制采样和系统遍历使用一个全面的拓扑路线图，确保合理覆盖的关键，但狭窄的区域低有效尺寸。然而，这本身是不够的构型熵和其他积分的准确计算，需要估计的自由能和动力学-其中，它是必不可少的，以获得均匀的采样在适当的carnival或模空间参数化。通过两个参数化之间的映射的雅可比矩阵对凯莱采样进行标准调整充满了来自病态雅可比矩阵的挑战。本文形式化和分析这些挑战，提供修改EASAL，确保凯莱采样的优势，同时确保一定的最小距离和覆盖范围之间的关系采样配置-在笛卡尔空间。将修饰的EASAL的性能与基本EASAL进行比较，并提供了人和大鼠胰岛淀粉样多肽（HiAPP、PDB-2KJ 7和RiAPP PDB-2KB 8）二聚化的数据（两者仅在37个残基中的6个残基上不同，但前者聚集成原纤维，而后者不）。
The development of tailored materials for specific applications is an active field of research in chemistry, material science and drug discovery. The number of possible molecules that can be obtained from a set of atomic species grow exponentially with the size of the system, limiting the efficiency of classical sampling algorithms. On the other hand, quantum computers can provide an efficient solution to the sampling of the chemical compound space for the optimization of a given molecular property. In this work we propose a quantum algorithm for addressing the material design problem with a f…
On the time scales accessible to atomistic numerical modelling, chemical reactions are considered rare events. Atomistic simulations are typically biased along a low-dimensional representation of a chemical reaction in an atomic structure space, i.e., along the collective variable, to accelerate sampling of these improbable events. However, suitable collective variables are often complicated to guess due to the complexity of the transitions. Therefore, we present an automatic method of generating robust collective variables from atomic representation vectors, using either fixed Behler-Parrinel…
Neural network (NN) interatomic potentials provide fast prediction of potential energy surfaces, closely matching the accuracy of the electronic structure methods used to produce the training data. However, NN predictions are only reliable within well-learned training domains, and show volatile behavior when extrapolating. Uncertainty quantification approaches can flag atomic configurations for which prediction confidence is low, but arriving at such uncertain regions requires expensive sampling of the NN phase space, often using atomistic simulations. Here, we exploit automatic differentiatio…
