Computational simulations play a central role in scientific discovery, and machine learning (ML) has emerged as a promising alternative to traditional physics-based modeling. However, scientific modeling requires physically meaningful predictions, raising a fundamental question for data-driven methods: to what extent can physical inductive biases-that is, prior assumptions about the structure of the physical world-emerge by learning from data alone? In this work, we study atomistic modeling, a representative field in the computational sciences where ML architectures have historically embedded strong physical inductive biases-such as geometric locality and graph structure. We develop a systematic understanding of how physical patterns can alternatively be discovered directly from data by training a model without domain-specific priors, including any manually defined atomistic pairwise interactions. We find that the model autonomously recovers key physical structure, such as learned interatomic interaction strengths that mirror classical electrostatics and interaction cutoffs consistent with traditional physical models. We further demonstrate predictable neural scaling law behavior with increased data and compute, and find accuracy on certain metrics competitive with physics-informed architectures. Our results clarify the boundary between engineered inductive biases and learnable physical structure, suggesting that general-purpose architectures can serve as principled baselines for scientific modeling by learning fundamental physical structure directly from data.
GraphXForm: Graph transformer for computer-aided molecular design
Technical University of Munich(慕尼黑工业大学);University of Applied Sciences Weihenstephan-Triesdorf(魏恩施蒂芬-特里斯多夫应用科学大学);RWTH Aachen University(亚琛工业大学);JARA Center for Simulation and Data Science(JARA模拟与数据科学中心);Forschungszentrum Jülich GmbH(于利希研究中心)
GeoT: A Geometry-aware Transformer for Reliable Molecular Property Prediction and Chemically Interpretable Representation Learning
Kakao Corporation(Kakao公司);LG Chem(LG化学);Seoul National University(首尔国立大学);Korea Advanced Institute of Science and Technology (KAIST)(韩国科学技术院(KAIST));Seoul National University of Science and Technology(首尔国立科技大学)
Triplet Interaction Improves Graph Transformers: Accurate Molecular Graph Learning with Triplet Graph Transformers
Department of Computer Science, Rensselaer Polytechnic Institute, Troy, NY, USA(理海理工学院计算机科学系);IBM Thomas J. Watson Research Center, Yorktown Heights, NY, USA(IBM 玛丽·沃森研究中心)
SMILES Transformer: Pre-trained Molecular Fingerprint for Low Data Drug Discovery
University of Tokyo(东京大学);International Research Center for Neurointelligence(国际神经智能研究中心);RIKEN Center for Biosystems Dynamics Research(理化学研究所生物系统动力学研究中心)
NEAT: Neighborhood-Guided, Efficient, Autoregressive Set Transformer for 3D Molecular Generation
Department of Pharmaceutical Sciences, University of Vienna(维也纳大学药学系);Christian Doppler Laboratory for Molecular Informatics in the Biosciences(生物医学分子信息学克里斯蒂安·多普勒实验室);Vienna Doctoral School of Pharmaceutical, Nutritional and Sport Sciences(维也纳药学、营养学和运动科学博士学院)
WGFormer: An SE(3)-Transformer Driven by Wasserstein Gradient Flows for Molecular Ground-State Conformation Prediction
Gaoling School of Artificial Intelligence, Renmin University of China(中国人民大学首都人工智能学院);Beijing Key Laboratory of Research on Large Models(北京大型模型研究关键实验室);Engineering Research Center of Next-Generation Intelligent Search(下一代智能搜索工程研究中心)
A Reinforcement Learning-Driven Transformer GAN for Molecular Generation
D3 Center(D3中心);Osaka University(大阪大学);Graduate School of Advanced Science and Engineering(先进科学与工程研究生院);Hiroshima University(广岛大学);Centre for Cyber Resilience and Trust(网络韧性与信任中心);Deakin University(德金大学);Graduate School of Informatics(信息科学研究生院);Nagoya University(名古屋大学)
Molformer: Motif-based Transformer on 3D Heterogeneous Molecular Graphs
School of Engineering, Westlake University(西湖大学工学院);Department of Computer Science, Yale University(耶鲁大学计算机科学系);Institute of AI Industry Research, Tsinghua University(清华大学人工智能产业研究院)
A systematic investigation of molecular encoding methods for drug property predictions across neural network and Transformer encoder-based model
School of Medicine, National Tsing Hua University(国立清华大学医学院);Institute of Bioinformatics and Structural Biology, National Tsing Hua University(国立清华大学生物信息学与结构生物学研究所);Department of Life Science, National Tsing Hua University(国立清华大学生命科学系);Interdisciplinary Program of Life Sciences and Medicine, National Tsing Hua University(国立清华大学生命科学与医学跨学科计划)
CSIR- Central Scientific Instruments Organisation(科学与工业研究理事会中央科学仪器组织);Academy of Scientific and Innovative Research (AcSIR)(科学与创新研究学院);Monell Chemical Senses Center(莫奈尔化学感官中心);University of Pennsylvania(宾夕法尼亚大学)
CandidateDrug4Cancer: An Open Molecular Graph Learning Benchmark on Drug Discovery for Cancer
Ping An Healthcare Technology(平安医疗科技);Chinese Academy of Medical Sciences(中国医学科学院);National Cancer Center(国家癌症中心);Xidian University School of Computer Science and Technology(西安电子科技大学计算机科学与技术学院)
Sampling 3D Molecular Conformers with Diffusion Transformers
Technical University Berlin(技术大学柏林);BIFOLD Berlin(柏林BIFOLD);Google DeepMind(谷歌DeepMind);MPI for Informatics, Saarbrücken(萨尔布吕肯信息研究所);Department of Artificial Intelligence, Korea University(韩国大学人工智能系)
An Equivariant Pretrained Transformer for Unified 3D Molecular Representation Learning
Tsinghua University(清华大学);Institute for AI Industry Research, Tsinghua University(清华大学人工智能产业研究院);Renmin University of China(中国人民大学);University of Pisa(比萨大学)