Learning effective protein representations is critical in a variety of tasks in biology such as predicting protein function or structure. Existing approaches usually pretrain protein language models on a large number of unlabeled amino acid sequences and then finetune the models with some labeled data in downstream tasks. Despite the effectiveness of sequence-based approaches, the power of pretraining on known protein structures, which are available in smaller numbers only, has not been explored for protein property prediction, though protein structures are known to be determinants of protein function. In this paper, we propose to pretrain protein representations according to their 3D structures. We first present a simple yet effective encoder to learn the geometric features of a protein. We pretrain the protein graph encoder by leveraging multiview contrastive learning and different self-prediction tasks. Experimental results on both function prediction and fold classification tasks show that our proposed pretraining methods outperform or are on par with the state-of-the-art sequence-based methods, while using much less pretraining data. Our implementation is available at https://github.com/DeepGraphLearning/GearNet.
Rigidity-Aware Geometric Pretraining for Protein Design and Conformational Ensembles
University of Illinois Urbana-Champaign(伊利诺伊大学厄巴纳-香槟分校);University of Washington(华盛顿大学);MPI for Intelligent Systems, Tübingen(智能系统马克斯·普朗克研究所,图宾根);National Research Council of Canada(加拿大国家研究理事会);University of Ottawa(渥太华大学);The Chinese University of Hong Kong(香港中文大学)
Zero-Shot Learning with Subsequence Reordering Pretraining for Compound-Protein Interaction
School of Computer Science, Wuhan University(武汉大学计算机学院);Engineering Research Center for Big Data Application in Private Health Medicine of Fujian Universities, Putian University(福建省私人健康医学大数据应用工程研究中心,莆田大学);Macquarie University(麦考瑞大学);Wuhan University Shenzhen Research Institute(武汉大学深圳研究院)
An Equivariant Pretrained Transformer for Unified 3D Molecular Representation Learning
Tsinghua University(清华大学);Institute for AI Industry Research, Tsinghua University(清华大学人工智能产业研究院);Renmin University of China(中国人民大学);University of Pisa(比萨大学)
Self-Conditioned Denoising for Atomistic Representation Learning
Department of Chemistry, Massachusetts Institute of Technology, 77 Massachusetts Avenue, Cambridge, 02139, MA, USA(麻省理工学院化学系);Department of Materials Science and Engineering, Massachusetts Institute of Technology, 77 Massachusetts Avenue, Cambridge, 02139, MA, USA(麻省理工学院材料科学与工程系)
学习有意义的蛋白质表示对于各种生物下游任务(如基于结构的药物设计)非常重要。在蛋白质序列预训练取得成功之后,对信息量更大的结构数据进行预训练成为一个很有前途的研究课题。然而,蛋白质结构预训练面临三大挑战:样本多样性不足,物理上不切实际的建模,以及缺乏蛋白质特定的借口任务。为了解决这些挑战,我们提出了3D几何预训练。在本文中,我们提出了一个统一的蛋白质预训练框架和一个基于3D几何的,数据高效的,蛋白质特定的借口任务:RefineDiff(Refine the Diffused Protein Structure Decoy)。在有限的数据(不到SOTA模型的1%)上对我们的几何感知模型进行预训练后,我们获得了信息丰富的蛋白质表示,可以在各种下游任务中实现相当的性能。
ProFSA: Self-supervised Pocket Pretraining via Protein Fragment-Surroundings Alignment
Institute for AI Industry Research, Tsinghua University(清华大学人工智能产业研究院);School of Information and Software Engineering, UESTC(电子科技大学信息与软件工程学院);Academy of Mathematics and Systems Science, Chinese Academy of Sciences(中国科学院数学与系统科学研究院)
Scaling Atomistic Protein Binder Design with Generative Pretraining and Test-Time Compute
NVIDIA(英伟达);University of Oxford(牛津大学);Mila - Québec AI Institute(米拉-魁北克人工智能研究所);Université de Montréal(蒙特利尔大学);HEC Montréal(蒙特利尔高等商学院);CIFAR AI Chair(CIFAR人工智能主席);AITHYRA;School of Biological Sciences, Seoul National University(首尔大学生物科学学院);Interdisciplinary Program in Bioinformatics, Seoul National University(首尔大学生物信息学跨学科项目);Institute of Molecular Biology and Genetics, Seoul National University(首尔大学分子生物学与遗传学研究所);Artificial Intelligence Institute, Seoul National University(首尔大学人工智能研究所)
RIGA-Fold: A General Framework for Protein Inverse Folding via Recurrent Interaction and Geometric Awareness
College of Computer Science and Software Engineering, Shenzhen University, Shenzhen, China(深圳大学计算机科学与软件工程学院);National Engineering Laboratory for Big Data System Computing Technology, Shenzhen University, Shenzhen, China(大数据系统计算技术国家工程实验室)
A Fine-tuning Dataset and Benchmark for Large Language Models for Protein Understanding
Toursun Synbio(图森合成生物);Johns Hopkins University(约翰斯·霍普金斯大学);University of Cambridge(剑桥大学);Shanghai Institute for Biomedical and Pharmaceutical Technologies(上海生物医药技术研究院);Shanghai AI Laboratory(上海人工智能实验室);Shanghai Jiao Tong University(上海交通大学);UNSW Syndey(新南威尔士大学悉尼分校)
Computer Network Information Center, Chinese Academy of Sciences(中国科学院计算机网络信息中心);Hangzhou Institute for Advanced Study, University of Chinese Academy of Sciences(中国科学院大学杭州高等研究院)
Generating Highly Designable Proteins with Geometric Algebra Flow Matching
Heidelberg Institute for Theoretical Studies(海德堡理论研究所);Heidelberg University(海德堡大学);Max Planck Institute for Polymer Research(马克斯·普朗克聚合物研究所);Stockholm University(斯德哥尔摩大学);Karlsruhe Institute of Technology(卡尔斯鲁厄理工学院)
Molecular Geometry Pretraining with SE(3)-Invariant Denoising Distance Matching
Mila - Québec AI Institute(米拉-魁北克人工智能研究所);Université de Montréal(蒙特利尔大学);National Research Council Canada(加拿大国家研究委员会);University of Ottawa(渥太华大学);HEC Montréal(蒙特利尔高等商学院)
From Supervision to Exploration: What Does Protein Language Model Learn During Reinforcement Learning?
The Chinese University of Hong Kong(香港中文大学);Peking University(北京大学);Stanford University(斯坦福大学);Nanjing University(南京大学);University of Pennsylvania(宾夕法尼亚大学);National University of Singapore(新加坡国立大学);University of Illinois Urbana-Champaign(伊利诺伊大学香槟分校);Toyota Technological Institute at Chicago(芝加哥丰田技术研究所)
An accurate nucleic acid-small molecule docking framework via geometric deep learning with large-scale pretraining
College of Pharmaceutical Sciences, Zhejiang University(浙江大学药学院);Faculty of Health Sciences, University of Macau(澳门大学健康科学学院);Zhejiang Provincial Key Laboratory for Intelligent Drug Discovery and Development, Jinhua Institute of Zhejiang University(浙江省智能药物发现与开发重点实验室,浙江大学金华研究院);Shanghai Innovation Institute(上海创新研究院)
One-step Structure Prediction and Screening for Protein-Ligand Complexes using Multi-Task Geometric Deep Learning
Medical School of Nanjing University(南京大学医学院);National Institute of Healthcare Data Science, Nanjing University(南京大学国家健康数据科学研究院);School of Life Sciences, Nanjing University(南京大学生命科学学院)