Department of Artificial Intelligence, Westlake University, Hangzhou, China(人工智能系,西湖大学,杭州,中国);Academy of Mathematics and Systems Science, Chinese Academy of Sciences, Beijing, China(数学与系统科学研究院,中国科学院,北京,中国)
Flow matching has shown state-of-the-art performance in various generative tasks, ranging from image generation to decision-making, where generation under energy guidance (abbreviated as guidance in the following) is pivotal. However, the guidance of flow matching is more general than and thus substantially different from that of its predecessor, diffusion models. Therefore, the challenge in guidance for general flow matching remains largely underexplored. In this paper, we propose the first framework of general guidance for flow matching. From this framework, we derive a family of guidance techniques that can be applied to general flow matching. These include a new training-free asymptotically exact guidance, novel training losses for training-based guidance, and two classes of approximate guidance that cover classical gradient guidance methods as special cases. We theoretically investigate these different methods to give a practical guideline for choosing suitable methods in different scenarios. Experiments on synthetic datasets, image inverse problems, and offline reinforcement learning demonstrate the effectiveness of our proposed guidance methods and verify the correctness of our flow matching guidance framework. Code to reproduce the experiments can be found at https://github.com/AI4Science-WestlakeU/flow_guidance.
Discrete Guidance Matching: Exact Guidance for Discrete Flow Matching
School of Statistics, East China Normal University(华东师范大学统计学院);Department of Statistics, University of California, Los Angeles(加州大学洛杉矶分校统计系);Department of Industrial and Systems Engineering, University of Minnesota(明尼苏达大学工业与系统工程系);School of Data Science, The Chinese University of Hong Kong, Shenzhen(香港中文大学(深圳)数据科学学院)
Gumbel-Softmax Flow Matching with Straight-Through Guidance for Controllable Biological Sequence Generation
Duke University(杜克大学);University of Pennsylvania(宾夕法尼亚大学);Duke-NUS Medical School(杜克-新加坡国立大学医学院);Mila, Quebec AI Institute(米拉魁北克人工智能研究所);Université de Montréal(蒙特利尔大学)
Value Gradient Guidance for Flow Matching Alignment
The Chinese University of Hong Kong (Shenzhen)(香港中文大学(深圳));University of Tübingen(图宾根大学);Microsoft Research(微软研究院);The Chinese University of Hong Kong(香港中文大学);Mila – Quebec AI Institute(魁北克AI研究所)
Alignment of Diffusion Model and Flow Matching for Text-to-Image Generation
Department of Statistics, University of California, Los Angeles(加州大学洛杉矶分校统计学系);Department of Industrial and Systems Engineering, University of Minneasota(明尼苏达大学工业与系统工程系);School of Data Science, The Chinese University of Hong Kong, Shenzhen(香港中文大学(深圳)数据科学学院)
Inference-Time Alignment for Diffusion Models via Variationally Stable Doob's Matching
Joint Laboratory of Data Science and Business Intelligence(数据科学与商业智能联合实验室);State Key Laboratory of Mathematical Sciences(数学科学国家重点实验室);Institut für Geometrie und Praktische Mathematik(几何与应用数学研究所);School of Artificial Intelligence(人工智能学院);School of Mathematics and Statistics(数学与统计学院)
Path-Guided Flow Matching for Dataset Distillation
Department of Machine Learning, MBZUAI, Abu Dhabi, UAE(机器学习系,MBZUAI,阿布扎赫德,阿联酋);Department of Computer Vision, MBZUAI, Abu Dhabi, UAE(计算机视觉系,MBZUAI,阿布扎赫德,阿联酋)