arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2402.05841q-bio.BMcs.LG
收藏

Dirichlet Flow Matching及其在DNA序列设计中的应用

Dirichlet Flow Matching with Applications to DNA Sequence Design

Massachusetts Institute of Technology(麻省理工学院);CSAIL(计算机科学与人工智能实验室)

Hannes Stark, Bowen Jing, Chenyu Wang, Gabriele Corso, Bonnie Berger, Regina Barzilay, Tommi Jaakkola

AI总结:

针对单纯形上线性流匹配的病态问题,提出基于Dirichlet分布混合的Dirichlet flow matching框架,支持分类器引导与无分类器引导,并通过蒸馏实现单步序列生成,在DNA序列设计任务中表现优异且大幅加速。

摘要:

离散扩散或流模型能够实现比自回归模型更快、更可控的序列生成。我们表明,在单纯形上简单的线性流匹配不足以实现这一目标,因为它在训练目标中存在不连续性以及进一步的病态问题。为了克服这一点,我们基于Dirichlet分布的混合作为概率路径,在单纯形上开发了Dirichlet flow matching。在此框架中,我们推导出混合分布的得分与流的向量场之间的联系,从而允许分类器引导和无分类器引导。此外,我们提供了蒸馏Dirichlet flow matching,它能够以最小的性能损失实现单步序列生成,与自回归模型相比实现了$O(L)$的加速。在复杂的DNA序列生成任务中,我们证明了在分布度量指标以及为生成的序列实现所需设计目标方面,相比所有基线模型具有更优的性能。最后,我们展示了我们的无分类器引导方法改善了无条件生成,并且对于生成满足设计目标的DNA是有效的。代码可在https://github.com/HannesStark/dirichlet-flow-matching获取。

英文摘要:

Discrete diffusion or flow models could enable faster and more controllable sequence generation than autoregressive models. We show that naïve linear flow matching on the simplex is insufficient toward this goal since it suffers from discontinuities in the training target and further pathologies. To overcome this, we develop Dirichlet flow matching on the simplex based on mixtures of Dirichlet distributions as probability paths. In this framework, we derive a connection between the mixtures' scores and the flow's vector field that allows for classifier and classifier-free guidance. Further, we provide distilled Dirichlet flow matching, which enables one-step sequence generation with minimal performance hits, resulting in $O(L)$ speedups compared to autoregressive models. On complex DNA sequence generation tasks, we demonstrate superior performance compared to all baselines in distributional metrics and in achieving desired design targets for generated sequences. Finally, we show that our classifier-free guidance approach improves unconditional generation and is effective for generating DNA that satisfies design targets. Code is available at https://github.com/HannesStark/dirichlet-flow-matching.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑