Despite Flow Matching and diffusion models having emerged as powerful generative paradigms for continuous variables such as images and videos, their application to high-dimensional discrete data, such as language, is still limited. In this work, we present Discrete Flow Matching, a novel discrete flow paradigm designed specifically for generating discrete data. Discrete Flow Matching offers several key contributions:(i) it works with a general family of probability paths interpolating between source and target distributions; (ii) it allows for a generic formula for sampling from these probability paths using learned posteriors such as the probability denoiser ($x$-prediction) and noise-prediction ($ε$-prediction); (iii) practically, focusing on specific probability paths defined with different schedulers improves generative perplexity compared to previous discrete diffusion and flow models; and (iv) by scaling Discrete Flow Matching models up to 1.7B parameters, we reach 6.7% Pass@1 and 13.4% Pass@10 on HumanEval and 6.7% Pass@1 and 20.6% Pass@10 on 1-shot MBPP coding benchmarks. Our approach is capable of generating high-quality discrete data in a non-autoregressive fashion, significantly closing the gap between autoregressive models and discrete flow models.
University of California, Irvine(加州大学伊万斯堡分校);University of Virginia(弗吉尼亚大学);University of Alabama at Birmingham(阿拉巴马大学伯明翰分校);Northeastern University(东北大学)
Discrete Guidance Matching: Exact Guidance for Discrete Flow Matching
School of Statistics, East China Normal University(华东师范大学统计学院);Department of Statistics, University of California, Los Angeles(加州大学洛杉矶分校统计系);Department of Industrial and Systems Engineering, University of Minnesota(明尼苏达大学工业与系统工程系);School of Data Science, The Chinese University of Hong Kong, Shenzhen(香港中文大学(深圳)数据科学学院)
Hierarchical Discrete Flow Matching for Graph Generation
Department of Computer Science, University of Geneva, Switzerland(瑞士日内瓦大学计算机科学系);DMML Group, Geneva School for Business administration HES-SO, Switzerland(瑞士日内瓦高等商学院DMML研究组)
Center for Foundation Models and Generative AI, Northwestern University(基础模型与生成式人工智能中心,西北大学);Department of Computer Science, Northwestern University(计算机科学系,西北大学);Department of Statistics and Data Science, Northwestern University(统计与数据科学系,西北大学)
Flow Matching for Offline Reinforcement Learning with Discrete Actions
Department of Computer Science, University of Kentucky(计算机科学系,肯塔基大学);Department of Computing and Information Sciences, Rochester Institute of Technology(计算与信息科学系,罗切斯特理工学院)
Department of Artificial Intelligence, Westlake University, Hangzhou, China(人工智能系,西湖大学,杭州,中国);Academy of Mathematics and Systems Science, Chinese Academy of Sciences, Beijing, China(数学与系统科学研究院,中国科学院,北京,中国)
Multi-Objective-Guided Discrete Flow Matching for Controllable Biological Sequence Design
Department of Biomedical Engineering, Duke University(杜克大学生物医学工程系);Department of Computer Science, Fudan University(复旦大学计算机科学系);Center of Computational Biology, Duke-NUS Medical School(杜克-新加坡国立大学医学学校计算生物学中心);Management and Technology Program, University of Pennsylvania(宾夕法尼亚大学管理与技术项目);Department of Computer Science, Duke University(杜克大学计算机科学系);Department of Biostatistics and Bioinformatics, Duke University(杜克大学生物统计与生物信息学系)
提出 MOG-DFM 框架,通过混合秩方向分数和自适应超锥滤波引导预训练离散流匹配模型,实现多目标帕累托高效的肽与增强子 DNA 序列设计。