Atom-JEPA：面向3D原子系统的联合嵌入预测架构
Atom-JEPA: Joint-Embedding Predictive Architecture for 3D Atomistic Systems
浏览论文内容
中文总结 AI 辅助
Atom-JEPA提出一种自监督预训练框架，通过原子级和子结构级目标从无标签3D结构学习潜在表示，在分子和晶体数据集上预训练，并在ADMET和量子化学属性预测任务上达到最先进性能，证明潜在空间预测预训练支持广泛下游泛化。
中文摘要 AI 辅助
大规模自监督预训练已经重塑了现代机器学习，显著提升了语言和视觉模型在下游任务中的泛化能力。尽管近年来深度学习在原子系统建模方面取得了显著进展，但该领域的自监督预训练尚未实现可比的跨下游任务泛化。为解决这一问题，我们提出了Atom-JEPA，一个自监督预训练框架，通过受联合嵌入预测架构启发的互补原子级和子结构级目标，从无标签的3D结构中学习潜在表示。我们在大规模分子和晶体数据集上预训练Atom-JEPA，并通过在多样化的下游属性预测任务上进行微调来评估其迁移性能。Atom-JEPA在分子ADMET和量子化学属性预测任务上达到了最先进的性能，并在预测晶体材料的物理属性方面极具竞争力。这些结果证明了潜在空间预测预训练仅从结构数据支持广泛下游泛化的潜力。代码和预训练模型检查点可在以下https URL公开获取。
英文摘要
Large-scale self-supervised pretraining has reshaped modern machine learning, substantially advancing the ability of language and vision models to generalize across downstream tasks. While deep learning has driven considerable progress in modeling atomistic systems in recent years, self-supervised pretraining in this domain has not yet achieved comparable downstream generalization. To address this, we introduce Atom-JEPA, a self-supervised pretraining framework that learns latent representations from unlabeled 3D structures through complementary atom-level and substructure-level objectives inspired by joint-embedding predictive architectures. We pretrain Atom-JEPA on large-scale molecular and crystalline datasets and evaluate its transfer performance by fine-tuning on a diverse set of downstream property prediction tasks. Atom-JEPA achieves state-of-the-art performance on molecular ADMET and quantum-chemical property prediction tasks, and is highly competitive in predicting the physical properties of crystalline materials. These results demonstrate the potential of latent-space predictive pretraining to support broad downstream generalization from structural data alone. Code and pretrained model checkpoints are publicly available at https://github.com/khelverskovp/atom-jepa
发表机构
- Technical University of Denmark（丹麦技术大学）
机构由 AI 辅助整理，请以论文原文为准。
