早期的DNA基础模型采用了BERT式训练， 在DNA理解任务上取得了良好的性能，但缺乏生成能力。最近的自回归模型能够实现DNA生成，但采用的是从左到右的因果建模，这对于具有内在双向调控关系的DNA来说效果不佳。我们提出了D3LM（Discrete DNA Diffusion Language Model），通过掩码扩散统一了双向表示学习和DNA生成。D3LM直接采用了核苷酸变换器（NT）v2架构，但将训练目标重新表述为离散DNA空间中的掩码扩散，使模型在单个模型中同时具备双向理解和生成能力。与相同规模的NT v2相比，D3LM在理解任务上取得了更好的性能。值得注意的是，在调控元件生成方面，D3LM实现了SFID为10.92，接近真实DNA序列（7.85），并大幅超越了先前自回归模型的最佳结果29.16。我们的工作表明扩散语言模型是统一DNA基础模型的一种有前景的范式。我们进一步对DNA领域中的掩码扩散模型进行了首次系统研究，探讨了诸如分词方案和采样策略等实用设计选择，从而提供了实证见解和坚实的基础，为未来研究奠定了基础。D3LM已发布在https://huggingface.co/collections/Hengchang-Liu/d3lm。
英文摘要：
Early DNA foundation models adopted BERT-style training, achieving good performance on DNA understanding tasks but lacking generative capabilities. Recent autoregressive models enable DNA generation, but employ left-to-right causal modeling that is suboptimal for DNA where regulatory relationships are inherently bidirectional. We present D3LM (\textbf{D}iscrete \textbf{D}NA \textbf{D}iffusion \textbf{L}anguage \textbf{M}odel), which unifies bidirectional representation learning and DNA generation through masked diffusion. D3LM directly adopts the Nucleotide Transformer (NT) v2 architecture but reformulates the training objective as masked diffusion in discrete DNA space, enabling both bidirectional understanding and generation capabilities within a single model. Compared to NT v2 of the same size, D3LM achieves improved performance on understanding tasks. Notably, on regulatory element generation, D3LM achieves an SFID of 10.92, closely approaching real DNA sequences (7.85) and substantially outperforming the previous best result of 29.16 from autoregressive models. Our work suggests diffusion language models as a promising paradigm for unified DNA foundation models. We further present the first systematic study of masked diffusion models in the DNA domain, investigating practical design choices such as tokenization schemes and sampling strategies, thereby providing empirical insights and a solid foundation for future research. D3LM has been released at https://huggingface.co/collections/Hengchang-Liu/d3lm.
JanusDNA: A Powerful Bi-directional Hybrid DNA Foundation Model
Berlin Institute of Health, Charité – Universitätsmedizin Berlin（柏林健康研究所，柏林夏里特大学医学中心）；College of Big Data and Internet, Shenzhen Technology University（大数据与互联网学院，深圳技术大学）；Language Technologies Institute, Carnegie Mellon University（语言技术研究所，卡内基梅隆大学）；Epigenetics Laboratory, Max Planck Institute for Heart and Lung Research（表观遗传学实验室，马克斯·普朗克心脏病和肺部研究所以）；Department of Mathematics and Computer Science, Freie Universität Berlin（数学与计算机科学系，柏林自由大学）；Health Data Science Unit, Heidelberg University Hospital and BioQuant（健康数据科学单元，海德堡大学医院和BioQuant）；Intelligent Medicine Institute, Fudan University（智能医学研究所，复旦大学）
SynBridge: Bridging Reaction States via Discrete Flow for Bidirectional Reaction Prediction
AI for Science Institute（人工智能科学研究院）；DP Technology（DP科技）；College of Chemistry and Molecular Engineering（化学与分子工程学院）；School of Mathematical Sciences（数学科学学院）；Center for Machine Learning Research（机器学习研究中心）
Enhancing DNA Foundation Models to Address Masking Inefficiencies
University of Waterloo（滑铁卢大学）；Vector Institute（向量研究所）；Simon Fraser University（西蒙菲莎大学）；Alberta Machine Intelligence Institute (Amii)（阿尔伯塔机器智能研究所）；University of Guelph（圭尔夫大学）
ShortListing Model: A Streamlined SimplexDiffusion for Discrete Variable Generation
Generative Symbolic Intelligence Lab (GenSI), Tsinghua University（生成符号智能实验室（GenSI），清华大学）；Institute for AI Industry Research (AIR), Tsinghua University（人工智能产业研究院（AIR），清华大学）
