Instruction-Based Molecular Graph Generation with Unified Text-Graph Diffusion Model
State Key Laboratory of General Artificial Intelligence, School of Intelligence Science and Technology, Peking University（通用人工智能国家重点实验室，智能科学与技术学院，北京大学）；State Key Laboratory of General Artificial Intelligence, School of Intelligence Science（通用人工智能国家重点实验室，智能科学学院）；The University of Hong Kong（香港大学）
Yuran Xiang, Haiteng Zhao, Chang Ma, Zhi-Hong Deng
Recent advancements in computational chemistry have increasingly focused on synthesizing molecules based on textual instructions. Integrating graph generation with these instructions is complex, leading most current methods to use molecular sequences with pre-trained large language models. In response to this challenge, we propose a novel framework, named $\textbf{UTGDiff (Unified Text-Graph Diffusion Model)}$, which utilizes language models for discrete graph diffusion to generate molecular graphs from instructions. UTGDiff features a unified text-graph transformer as the denoising network, derived from pre-trained language models and minimally modified to process graph data through attention bias. Our experimental results demonstrate that UTGDiff consistently outperforms sequence-based baselines in tasks involving instruction-based molecule generation and editing, achieving superior performance with fewer parameters given an equivalent level of pretraining corpus. Our code is availble at https://github.com/ran1812/UTGDiff.
3M-Diffusion: Latent Multi-Modal Diffusion for Language-Guided Molecular Structure Generation
Artificial Intelligence Research Laboratory（人工智能研究实验室）；College of Information Sciences and Technology（信息科学与技术学院）；Center for Artificial Intelligence Foundations and Scientific Applications（人工智能基础与科学应用中心）；Institute for Computational and Data Sciences（计算与数据科学研究所）；The Pennsylvania State University（宾夕法尼亚州立大学）
LLaDA-V: Large Language Diffusion Models with Visual Instruction Tuning
Gaoling School of AI, Renmin University of China（中国人民大学人工智能学院）；Beijing Key Laboratory of Research on Large Models and Intelligent Governance（北京大型模型与智能治理研究重点实验室）；Engineering Research Center of Next-Generation Intelligent Search and Recommendation, MOE（下一代智能搜索与推荐工程技术研究中心，教育部）；Ant Group（蚂蚁集团）
We propose a method for editing images from human instructions: given an input image and a written instruction that tells the model what to do, our model follows these instructions to edit the image. To obtain training data for this problem, we combine the knowledge of two large pretrained models -- a language model (GPT-3) and a text-to-image model (Stable Diffusion) -- to generate a large dataset of image editing examples. Our conditional diffusion model, InstructPix2Pix, is trained on our generated data, and generalizes to real images and user-written instructions at inference time. Since i…
Since its foundations, more than one hundred years ago, the field of structural biology has strived to understand and analyze the properties of molecules and their interactions by studying the structure that they take in 3D space. However, a fundamental challenge with this approach has been the dynamic nature of these particles, which forces us to model not a single but a whole distribution of structures for every molecular system. This thesis proposes Intrinsic Diffusion Modeling, a novel approach to this problem based on combining diffusion generative models with scientific knowledge about t…
Uni-Instruct: One-step Diffusion Model through Unified Diffusion Divergence Instruction
College of Future Technology（未来技术学院）；National Biomedical Imaging Center（国家生物医学成像中心）；Academy for Advanced Interdisciplinary Studies（先进跨学科研究学院）；hi-lab；Yuanpei College（元培学院）
Frame-based Equivariant Diffusion Models for 3D Molecular Generation
University of Amsterdam（阿姆斯特丹大学）；AMLab；AI4Science Lab（AI4Science实验室）；Korteweg-de Vries Institute for Mathematics University of Amsterdam（Korteweg-de Vries数学研究所阿姆斯特丹大学）
Equivariant Asynchronous Diffusion: An Adaptive Denoising Schedule for Accelerated Molecular Conformation Generation
Shanghai Academy of Artificial Intelligence for Science, SAIS（上海人工智能科学研究院，SAIS）；College of Chemistry and Chemical Engineering, Xiamen University（厦门大学化学与化工学院）；Artificial Intelligence Innovation and Incubation (AI 3 ) Institute, Fudan University（复旦大学人工智能创新与孵化（AI 3）研究院）
Straight-Line Diffusion Model for Efficient 3D Molecular Generation
Academy of Mathematics and Systems Science, Chinese Academy of Sciences（中国科学院数学与系统科学研究院）；Institute for AI Industry Research (AIR), Tsinghua University（清华大学人工智能产业研究院）；University of Chinese Academy of Sciences（中国科学院大学）；Huazhong University of Science and Technology（华中科技大学）；Beijing Academy of Artificial Intelligence（北京人工智能研究院）
Massachusetts Institute of Technology（麻省理工学院）；Harvard University（哈佛大学）
Molecular conformer generation is a fundamental task in computational chemistry. Several machine learning approaches have been developed, but none have outperformed state-of-the-art cheminformatics methods. We propose torsional diffusion, a novel diffusion framework that operates on the space of torsion angles via a diffusion process on the hypertorus and an extrinsic-to-intrinsic score model. On a standard benchmark of drug-like molecules, torsional diffusion generates superior conformer ensembles compared to machine learning and cheminformatics methods in terms of both RMSD and chemical prop…
Towards 3D Molecule-Text Interpretation in Language Models
University of Science and Technology of China（中国科学技术大学）；National University of Singapore（新加坡国立大学）；Institute of Dataspace, Hefei Comprehensive National Science Center（合肥综合性国家科学中心数据空间研究所）；Huawei Cloud（华为云）
