PromptCharm：通过多模态提示词构建与优化实现文本生成图像
PromptCharm: Text-to-Image Generation through Multi-modal Prompting and Refinement
AI总结：
针对新手构建文本生成图像提示词难的问题，提出混合主动式系统PromptCharm，通过自动优化提示词、风格库选择、注意力可视化及反馈式图像调整提升创作效果，用户研究验证其有效性。
摘要：
生成式人工智能的最新进展极大推动了文本生成图像领域的发展。当前最先进的文本生成图像模型Stable Diffusion（稳定扩散模型）已能够合成具备高度美感的高质量图像。因此，构建出既符合模型解读逻辑、又匹配用户意图的文本提示词就变得至关重要。然而，由于Stable Diffusion模型的复杂性，以及迭代编辑和优化文本提示词需要付出大量精力，对于新手用户来说，构建提示词仍然颇具挑战。为解决这些难题，我们提出了PromptCharm——一套混合主动式系统，通过多模态提示词工程与优化助力文本生成图像创作。为帮助新手用户构建提示词，PromptCharm首先会自动优化和完善用户的初始提示词。此外，PromptCharm支持用户在大型数据库中探索和选择不同的图像风格。为帮助用户有效优化提示词与图像，PromptCharm通过可视化模型的注意力值来呈现模型的解释逻辑。如果用户发现生成的图像存在任何不满意的区域，可在PromptCharm的丰富反馈循环中，通过模型注意力调整或图像修复进一步优化图像。为评估PromptCharm的有效性与可用性，我们开展了一项包含12名参与者的对照用户研究，以及另一项包含12名参与者的探索性用户研究。两项研究表明，与使用缺少交互或可视化支持的两个PromptCharm变体版本相比，使用PromptCharm的参与者能够创作出质量更高、更符合用户预期的图像。
英文摘要：
The recent advancements in Generative AI have significantly advanced the field of text-to-image generation. The state-of-the-art text-to-image model, Stable Diffusion, is now capable of synthesizing high-quality images with a strong sense of aesthetics. Crafting text prompts that align with the model's interpretation and the user's intent thus becomes crucial. However, prompting remains challenging for novice users due to the complexity of the stable diffusion model and the non-trivial efforts required for iteratively editing and refining the text prompts. To address these challenges, we propose PromptCharm, a mixed-initiative system that facilitates text-to-image creation through multi-modal prompt engineering and refinement. To assist novice users in prompting, PromptCharm first automatically refines and optimizes the user's initial prompt. Furthermore, PromptCharm supports the user in exploring and selecting different image styles within a large database. To assist users in effectively refining their prompts and images, PromptCharm renders model explanations by visualizing the model's attention values. If the user notices any unsatisfactory areas in the generated images, they can further refine the images through model attention adjustment or image inpainting within the rich feedback loop of PromptCharm. To evaluate the effectiveness and usability of PromptCharm, we conducted a controlled user study with 12 participants and an exploratory user study with another 12 participants. These two studies show that participants using PromptCharm were able to create images with higher quality and better aligned with the user's expectations compared with using two variants of PromptCharm that lacked interaction or visualization support.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
