arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2403.16995cs.CLcs.AIcs.LGstat.ML
收藏

Language Rectified Flow:以概率流推进扩散语言生成

Language Rectified Flow: Advancing Diffusion Language Generation with Probabilistic Flows

The University of Texas at Austin(德克萨斯大学奥斯汀分校)

Shujian Zhang, Lemeng Wu, Chengyue Gong, Xingchao Liu

AI总结:

本文提出Language Rectified Flow,通过重新表述标准概率流模型并学习常微分方程在源分布与目标分布间传输,有效减少推理时间,在细粒度控制与文本编辑任务上持续优于基线。

摘要:

近期的研究已经证明,基于扩散语言模型在控制句子属性(如情感)和结构(如句法结构)方面取得了成功。驱动从噪声中生成高质量样本的卓越性能的一个关键组件是进行数千步的迭代去噪。虽然这有益处,但从噪声开始以及学习步骤的复杂性限制了其在许多现实世界NLP应用中的实施。本文提出了Language Rectified Flow。我们的方法基于对标准概率流模型的重新表述。Language Rectified Flow学习(神经)常微分方程模型,以在源分布和目标分布之间进行传输,从而为生成建模和领域迁移提供统一且有效的解决方案。从源分布出发,我们的Language Rectified Flow实现了快速模拟并有效减少了推理时间。在三个具有挑战性的细粒度控制任务和多个高质量文本编辑上的实验表明,我们的方法持续优于其基线。广泛的实验和消融研究表明,我们的方法对于许多NLP任务具有通用性、有效性且大有裨益。

英文摘要:

Recent works have demonstrated success in controlling sentence attributes ($e.g.$, sentiment) and structure ($e.g.$, syntactic structure) based on the diffusion language model. A key component that drives theimpressive performance for generating high-quality samples from noise is iteratively denoise for thousands of steps. While beneficial, the complexity of starting from the noise and the learning steps has limited its implementation to many NLP real-world applications. This paper proposes Language Rectified Flow ({\ours}). Our method is based on the reformulation of the standard probabilistic flow models. Language rectified flow learns (neural) ordinary differential equation models to transport between the source distribution and the target distribution, hence providing a unified and effective solution to generative modeling and domain transfer. From the source distribution, our language rectified flow yields fast simulation and effectively decreases the inference time. Experiments on three challenging fine-grained control tasks and multiple high-quality text editing show that our method consistently outperforms its baselines. Extensive experiments and ablation studies demonstrate that our method can be general, effective, and beneficial for many NLP tasks.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑