Potts models and variational autoencoders (VAEs) have recently gained popularity as generative protein sequence models (GPSMs) to explore fitness landscapes and predict the effect of mutations. Despite encouraging results, quantitative characterization and comparison of GPSM-generated probability distributions is still lacking. It is currently unclear whether GPSMs can faithfully reproduce the complex multi-residue mutation patterns observed in natural sequences arising due to epistasis. We develop a set of sequence statistics to assess the "generative capacity" of three GPSMs of recent interest: the pairwise Potts Hamiltonian, the VAE, and the site-independent model, using natural and synthetic datasets. We show that the generative capacity of the Potts Hamiltonian model is the largest, in that the higher order mutational statistics generated by the model agree with those observed for natural sequences. In contrast, we show that the VAE's generative capacity lies between the pairwise Potts and site-independent models. Importantly, our work measures GPSM generative capacity in terms of higher-order sequence covariation statistics which we have developed, and provides a new framework for evaluating and interpreting GPSM accuracy that emphasizes the role of epistasis.
The paper addresses the problem of finding the causal direction between two associated variables. The proposed solution is to build an autoencoder of their joint distribution and to maximize its estimation capacity relative to both the marginal distributions. It is shown that the resulting two capacities cannot, in general, be equal. This leads to a new criterion for causal discovery: the higher capacity is consistent with the unconstrained choice of a distribution representing the cause while the lower capacity reflects the constraints imposed by the mechanism on the distribution of the effec…
From thermodynamics to protein design: Diffusion models for biomolecule generation towards autonomous protein engineering
University of Paris City（巴黎城市大学）；University of Reunion（留尼汪大学）；INSERM（法国国家健康与医学研究院）；Imperial College London（伦敦帝国学院）；University of Magallanes（麦哲伦大学）；Centre for Biotechnology and Bioengineering, Universidad de Chile（智利大学生物技术与生物工程中心）；Leibniz Institute of Plant Biochemistry（莱布尼茨植物生物化学研究所）；National Centre for Biological Science, TIFR（塔塔基础研究院国家生物科学中心）；Indian Institute of Science（印度科学学院）；Institute of Bioinformatics and Applied Biotechnology（生物信息学与应用生物技术研究所）；EnergyLab（能源实验室）；Faculty of Sciences and Technology, University of Reunion（留尼汪大学科学与技术学院）；School of Information Science and Technology（信息科学与技术学院）；Beijing Institute of Artificial Intelligence（北京人工智能研究院）；University Le Havre Normandie（勒阿弗尔诺曼底大学）；PEACCEL（PEACCEL公司）；AI for Biologics（AI for Biologics部门）
Probabilistic circuits (PCs) are a tractable representation of probability distributions allowing for exact and efficient computation of likelihoods and marginals. There has been significant recent progress on improving the scale and expressiveness of PCs. However, PC training performance plateaus as model size increases. We discover that most capacity in existing large PC structures is wasted: fully-connected parameter layers are only sparsely used. We propose two operations: pruning and growing, that exploit the sparsity of PC structures. Specifically, the pruning operation removes unimporta…
Predicting mutational effects on protein-protein binding via a side-chain diffusion probabilistic model
Institute of Computing Technology, Chinese Academy of Sciences（中国科学院计算技术研究所）；University of Chinese Academy of Sciences（中国科学院大学）；Central China Research Institute for Artificial Intelligence Technologies（中原人工智能产业技术研究院）
Massachusetts Institute of Technology（麻省理工学院）；University of Washington（华盛顿大学）
Construction of a scaffold structure that supports a desired motif, conferring protein function, shows promise for the design of vaccines and enzymes. But a general solution to this motif-scaffolding problem remains open. Current machine-learning techniques for scaffold design are either limited to unrealistically small scaffolds (up to length 20) or struggle to produce multiple diverse scaffolds. We propose to learn a distribution over diverse and longer protein backbone structures via an E(3)-equivariant graph neural network. We develop SMCDiff to efficiently sample scaffolds from this distr…
Boosting Convolutional Neural Networks' Protein Binding Site Prediction Capacity Using SE(3)-invariant transformers, Transfer Learning and Homology-based Augmentation
Pre-trained protein language models have demonstrated significant applicability in different protein engineering task. A general usage of these pre-trained transformer models latent representation is to use a mean pool across residue positions to reduce the feature dimensions to further downstream tasks such as predicting bio-physics properties or other functional behaviours. In this paper we provide a two-fold contribution to machine learning (ML) driven drug design. Firstly, we demonstrate the power of sparsity by promoting penalization of pre-trained transformer models to secure more robust…
From Supervision to Exploration: What Does Protein Language Model Learn During Reinforcement Learning?
The Chinese University of Hong Kong（香港中文大学）；Peking University（北京大学）；Stanford University（斯坦福大学）；Nanjing University（南京大学）；University of Pennsylvania（宾夕法尼亚大学）；National University of Singapore（新加坡国立大学）；University of Illinois Urbana-Champaign（伊利诺伊大学香槟分校）；Toyota Technological Institute at Chicago（芝加哥丰田技术研究所）
Predicting clinical significance of BRCA1 and BRCA2 single nucleotide substitution variants with unknown clinical significance using probabilistic neural network and deep neural network-stacked autoencoder
University of Zanjan（赞詹大学）；Zanjan University of Medical Sciences（赞詹医科大学）
Mask prior-guided denoising diffusion improves inverse protein folding
School of Computer Science, University of Sheffield（谢菲尔德大学计算机科学学院）；Biologics Engineering, Oncology R&D, AstraZeneca（阿斯利康生物制药工程与肿瘤研发部）；Medicinal Chemistry, Research and Early Development, Cardiovascular, Renal and Metabolism, BioPharmaceuticals R&D, AstraZeneca（阿斯利康医药化学与研发部）；School of Biological Sciences, University of Southampton（南安普顿大学生物科学学院）；Centre for Machine Intelligence, University of Sheffield（谢菲尔德大学智能中心）
