Diffusion models have emerged as powerful tools for generative tasks, producing high-quality outputs across diverse domains. However, how the generated data responds to the initial noise perturbation in diffusion models remains under-explored, which hinders understanding the controllability of the sampling process. In this work, we first observe an interesting phenomenon: the relationship between the change of generation outputs and the scale of initial noise perturbation is highly linear through the diffusion ODE sampling. Then we provide both theoretical and empirical study to justify this linearity property of this input-output (noise-generation data) relationship. Inspired by these new insights, we propose a novel Controllable and Constrained Sampling method (CCS) together with a new controller algorithm for diffusion models to sample with desired statistical properties while preserving good sample quality. We perform extensive experiments to compare our proposed sampling approach with other methods on both sampling controllability and sampled data quality. Results show that our CCS method achieves more precisely controlled sampling while maintaining superior sample quality and diversity.
Fast Controllable Diffusion Models for Undersampled MRI Reconstruction
School of Electrical Engineering and Computer Science, University of Queensland(昆士兰大学电气工程与计算机科学学院);School of Mathematics and Physics, University of Queensland(昆士兰大学数学与物理学院)
Matrix Completion with Cross-Concentrated Sampling: Bridging Uniform Sampling and CUR Sampling
University of Central Florida(中佛罗里达大学);Michigan State University(密歇根州立大学);University of Michigan(密歇根大学);University of California, Los Angeles(加州大学洛杉矶分校)
MaRS: A Fast Sampler for Mean Reverting Diffusion based on ODE and SDE Solvers
School of Artificial Intelligence, University of Chinese Academy of Sciences(中国科学院大学人工智能学院);Institute of Automation, Chinese Academy of Sciences (CASIA)(中国科学院自动化研究所);DAMO Academy, Alibaba Group(阿里巴巴达摩院);Hupan Laboratory(湖畔实验室)
Mix and Match: Learning-free Controllable Text Generation using Energy Language Models
University of California San Diego(加州大学圣迭戈分校);Toyota Technological Institute at Chicago (TTIC)(芝加哥丰田技术研究院(TTIC))
控制文本生成的最新工作要么需要基于属性的微调的基础语言模型(LM),或限制了参数化的属性映射,以兼容的基础自回归LM。在这项工作中,我们提出了Mix and Match LM,这是一种基于全局分数的可控文本生成方法,它结合了任意预训练的黑盒模型,用于在生成的文本中实现所需的属性,而不涉及任何关于黑盒模型的微调或结构假设。我们将可控生成的任务解释为从基于能量的模型中抽取样本,该模型的能量值是来自黑盒模型的分数的线性组合,黑盒模型分别负责流畅性、控制属性和对任何条件上下文的忠诚度。我们使用Metropolis-Hastings抽样方案从这个基于能量的模型中使用双向上下文和全局属性特征进行抽样。我们验证了我们的方法在各种控制生成和基于风格的文本修订任务的有效性,优于最近提出的方法,涉及额外的训练,微调,或限制性的假设模型的形式。
Multi-Objective-Guided Discrete Flow Matching for Controllable Biological Sequence Design
Department of Biomedical Engineering, Duke University(杜克大学生物医学工程系);Department of Computer Science, Fudan University(复旦大学计算机科学系);Center of Computational Biology, Duke-NUS Medical School(杜克-新加坡国立大学医学学校计算生物学中心);Management and Technology Program, University of Pennsylvania(宾夕法尼亚大学管理与技术项目);Department of Computer Science, Duke University(杜克大学计算机科学系);Department of Biostatistics and Bioinformatics, Duke University(杜克大学生物统计与生物信息学系)
提出 MOG-DFM 框架,通过混合秩方向分数和自适应超锥滤波引导预训练离散流匹配模型,实现多目标帕累托高效的肽与增强子 DNA 序列设计。
Robust Low-Tubal-Rank Tensor Completion under Cross-Concentrated Sampling
University of Central Florida(中佛罗里达大学);Michigan State University(密歇根州立大学);University of Kentucky(肯塔基大学);University of Texas MD Anderson Cancer Center(德克萨斯大学 MD 安德森癌症中心)
McGill University(麦吉尔大学);Valence Labs(Valence实验室);Mila - Québec AI Institute(米拉-魁北克人工智能研究所);Google Deepmind(谷歌DeepMind);Hong Kong University of Science and Technology(香港科技大学)
DocDjinn: Controllable Synthetic Document Generation with VLMs and Handwriting Diffusion
RheinMain University of Applied Sciences(莱茵-美因应用科学大学);German Research Center for Artificial Intelligence(德国人工智能研究中心);DeepReader GmbH(DeepReader公司);Insiders Technologies GmbH(Insiders Technologies公司);National University of Sciences and Technology(国家科学与技术大学)