迈向基于无条件扩散模型的实时文本驱动图像操控
Towards Real-time Text-driven Image Manipulation with Unconditional Diffusion Models
AI总结：
本文提出一种基于无条件扩散模型的高效文本驱动图像操控算法，学习速度提升4.5-10倍、应用速度提升8倍，且仅需4秒即可动态适配用户指定的图像和文本，在保证高质量的同时显著降低了计算成本。
摘要：
扩散模型的最新进展为图像编辑提供了许多强大的工具。其中之一是文本驱动的图像操控：根据提供的文本描述编辑图像的语义属性。现有的基于扩散的方法已经能够针对广泛的文本提示实现高质量的图像操控。然而，在实际应用中，即使使用高端GPU，这些方法也需要高昂的计算成本。这极大地限制了基于扩散的图像编辑在现实世界中的潜在应用，尤其是在用户设备上运行时。在本文中，我们探讨了近期基于无条件扩散模型的文本驱动编辑方法的效率，并开发了一种新算法，该算法学习图像操控的速度快4.5到10倍，应用速度也快8倍。我们使用人工标注者在多个数据集上仔细评估了所提方法的视觉质量和表现力。我们的实验表明，我们的算法达到了比其昂贵得多的方法的质量。最后，我们展示了我们的方法仅需4秒即可将预训练模型动态适配到用户指定的图像和文本描述。在这种设置下，我们注意到更紧凑的无条件扩散模型可以被视为流行的文本条件对应模型的合理替代方案。
英文摘要：
Recent advances in diffusion models enable many powerful instruments for image editing. One of these instruments is text-driven image manipulations: editing semantic attributes of an image according to the provided text description. % Popular text-conditional diffusion models offer various high-quality image manipulation methods for a broad range of text prompts. Existing diffusion-based methods already achieve high-quality image manipulations for a broad range of text prompts. However, in practice, these methods require high computation costs even with a high-end GPU. This greatly limits potential real-world applications of diffusion-based image editing, especially when running on user devices. In this paper, we address efficiency of the recent text-driven editing methods based on unconditional diffusion models and develop a novel algorithm that learns image manipulations 4.5-10 times faster and applies them 8 times faster. We carefully evaluate the visual quality and expressiveness of our approach on multiple datasets using human annotators. Our experiments demonstrate that our algorithm achieves the quality of much more expensive methods. Finally, we show that our approach can adapt the pretrained model to the user-specified image and text description on the fly just for 4 seconds. In this setting, we notice that more compact unconditional diffusion models can be considered as a rational alternative to the popular text-conditional counterparts.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
