编辑：用于文本到图像扩散模型的有效且可解释的提示倒置
EDITOR: Effective and Interpretable Prompt Inversion for Text-to-Image Diffusion Models
AI总结：
本文提出\sys技术，通过预训练模型初始化、潜在空间反向工程和嵌入到文本转换，提升文本到图像扩散模型的提示倒置效果，实现更高的图像相似性和可解释性。
摘要：
文本到图像生成模型（例如，Stable Diffusion）已取得显著进展，能够根据文本描述生成高质量且逼真的图像。提示倒置，即识别生成特定制品所使用的文本提示，对于数据归属、模型来源和水印验证等应用具有重要潜力。最近的研究引入了延迟投影方案来优化代表词汇空间的提示，但仍然存在语义流畅性和效率方面的挑战。先进的图像描述模型或视觉大语言模型可以生成高度可解释的提示，但它们通常在图像相似性方面有所欠缺。在本文中，我们提出了一种名为\sys的提示倒置技术，用于文本到图像扩散模型，包括使用预训练的图像描述模型初始化嵌入，通过反向工程在潜在空间中进行细化，并通过嵌入到文本模型将其转换为文本。我们在广泛使用的数据集上进行了实验，如MS COCO、LAION、Flickr和DiffusionDB，结果显示，我们的方法在图像相似性、文本对齐、提示可解释性和通用性方面均优于现有方法。我们进一步展示了所生成提示在跨概念图像合成、概念操控、进化多概念生成和无监督分割等任务中的应用。
英文摘要：
Text-to-image generation models~(e.g., Stable Diffusion) have achieved significant advancements, enabling the creation of high-quality and realistic images based on textual descriptions. Prompt inversion, the task of identifying the textual prompt used to generate a specific artifact, holds significant potential for applications including data attribution, model provenance, and watermarking validation. Recent studies introduced a delayed projection scheme to optimize for prompts representative of the vocabulary space, though challenges in semantic fluency and efficiency remain. Advanced image captioning models or visual large language models can generate highly interpretable prompts, but they often lack in image similarity. In this paper, we propose a prompt inversion technique called \sys for text-to-image diffusion models, which includes initializing embeddings using a pre-trained image captioning model, refining them through reverse-engineering in the latent space, and converting them to texts using an embedding-to-text model. Our experiments on the widely-used datasets, such as MS COCO, LAION, Flickr and DiffusionDB, show that our method outperforms existing methods in terms of image similarity, textual alignment, prompt interpretability and generalizability. We further illustrate the application of our generated prompts in tasks such as cross-concept image synthesis, concept manipulation, evolutionary multi-concept generation and unsupervised segmentation.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
