Unlike vision and language data which usually has a unique format, molecules can naturally be characterized using different chemical formulations. One can view a molecule as a 2D graph or define it as a collection of atoms located in a 3D space. For molecular representation learning, most previous works designed neural networks only for a particular data format, making the learned models likely to fail for other data formats. We believe a general-purpose neural network model for chemistry should be able to handle molecular tasks across data modalities. To achieve this goal, in this work, we develop a novel Transformer-based Molecular model called Transformer-M, which can take molecular data of 2D or 3D formats as input and generate meaningful semantic representations. Using the standard Transformer as the backbone architecture, Transformer-M develops two separated channels to encode 2D and 3D structural information and incorporate them with the atom features in the network modules. When the input data is in a particular format, the corresponding channel will be activated, and the other will be disabled. By training on 2D and 3D molecular data with properly designed supervised signals, Transformer-M automatically learns to leverage knowledge from different data modalities and correctly capture the representations. We conducted extensive experiments for Transformer-M. All empirical results show that Transformer-M can simultaneously achieve strong performance on 2D and 3D tasks, suggesting its broad applicability. The code and models will be made publicly available at https://github.com/lsj2408/Transformer-M.
GeoT: A Geometry-aware Transformer for Reliable Molecular Property Prediction and Chemically Interpretable Representation Learning
Kakao Corporation（Kakao公司）；LG Chem（LG化学）；Seoul National University（首尔国立大学）；Korea Advanced Institute of Science and Technology (KAIST)（韩国科学技术院（KAIST））；Seoul National University of Science and Technology（首尔国立科技大学）
An Equivariant Pretrained Transformer for Unified 3D Molecular Representation Learning
Tsinghua University（清华大学）；Institute for AI Industry Research, Tsinghua University（清华大学人工智能产业研究院）；Renmin University of China（中国人民大学）；University of Pisa（比萨大学）
SMILES Transformer: Pre-trained Molecular Fingerprint for Low Data Drug Discovery
University of Tokyo（东京大学）；International Research Center for Neurointelligence（国际神经智能研究中心）；RIKEN Center for Biosystems Dynamics Research（理化学研究所生物系统动力学研究中心）
Triplet Interaction Improves Graph Transformers: Accurate Molecular Graph Learning with Triplet Graph Transformers
Department of Computer Science, Rensselaer Polytechnic Institute, Troy, NY, USA（理海理工学院计算机科学系）；IBM Thomas J. Watson Research Center, Yorktown Heights, NY, USA（IBM 玛丽·沃森研究中心）
NEAT: Neighborhood-Guided, Efficient, Autoregressive Set Transformer for 3D Molecular Generation
Department of Pharmaceutical Sciences, University of Vienna（维也纳大学药学系）；Christian Doppler Laboratory for Molecular Informatics in the Biosciences（生物医学分子信息学克里斯蒂安·多普勒实验室）；Vienna Doctoral School of Pharmaceutical, Nutritional and Sport Sciences（维也纳药学、营养学和运动科学博士学院）
GraphXForm: Graph transformer for computer-aided molecular design
Technical University of Munich（慕尼黑工业大学）；University of Applied Sciences Weihenstephan-Triesdorf（魏恩施蒂芬-特里斯多夫应用科学大学）；RWTH Aachen University（亚琛工业大学）；JARA Center for Simulation and Data Science（JARA模拟与数据科学中心）；Forschungszentrum Jülich GmbH（于利希研究中心）
Molformer: Motif-based Transformer on 3D Heterogeneous Molecular Graphs
School of Engineering, Westlake University（西湖大学工学院）；Department of Computer Science, Yale University（耶鲁大学计算机科学系）；Institute of AI Industry Research, Tsinghua University（清华大学人工智能产业研究院）
CSIR- Central Scientific Instruments Organisation（科学与工业研究理事会中央科学仪器组织）；Academy of Scientific and Innovative Research (AcSIR)（科学与创新研究学院）；Monell Chemical Senses Center（莫奈尔化学感官中心）；University of Pennsylvania（宾夕法尼亚大学）
WGFormer: An SE(3)-Transformer Driven by Wasserstein Gradient Flows for Molecular Ground-State Conformation Prediction
Gaoling School of Artificial Intelligence, Renmin University of China（中国人民大学首都人工智能学院）；Beijing Key Laboratory of Research on Large Models（北京大型模型研究关键实验室）；Engineering Research Center of Next-Generation Intelligent Search（下一代智能搜索工程研究中心）
Sampling 3D Molecular Conformers with Diffusion Transformers
Technical University Berlin（技术大学柏林）；BIFOLD Berlin（柏林BIFOLD）；Google DeepMind（谷歌DeepMind）；MPI for Informatics, Saarbrücken（萨尔布吕肯信息研究所）；Department of Artificial Intelligence, Korea University（韩国大学人工智能系）
A Reinforcement Learning-Driven Transformer GAN for Molecular Generation
D3 Center（D3中心）；Osaka University（大阪大学）；Graduate School of Advanced Science and Engineering（先进科学与工程研究生院）；Hiroshima University（广岛大学）；Centre for Cyber Resilience and Trust（网络韧性与信任中心）；Deakin University（德金大学）；Graduate School of Informatics（信息科学研究生院）；Nagoya University（名古屋大学）
A systematic investigation of molecular encoding methods for drug property predictions across neural network and Transformer encoder-based model
School of Medicine, National Tsing Hua University（国立清华大学医学院）；Institute of Bioinformatics and Structural Biology, National Tsing Hua University（国立清华大学生物信息学与结构生物学研究所）；Department of Life Science, National Tsing Hua University（国立清华大学生命科学系）；Interdisciplinary Program of Life Sciences and Medicine, National Tsing Hua University（国立清华大学生命科学与医学跨学科计划）
Transformers for molecular property prediction: Domain adaptation efficiently improves performance
Spoken Language Systems, Saarland Informatics Campus, Saarland University（spoken language systems，萨尔兰信息学校园，萨尔兰大学）；Medical Faculty, Saarland University（医学学院，萨尔兰大学）；Center for Bioinformatics, Saarland University（生物信息学中心，萨尔兰大学）；Data Driven Drug Design, Center for Bioinformatics, Saarland University（数据驱动药物设计，生物信息学中心，萨尔兰大学）
