arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2407.08513cs.CV
收藏

用于风格化图标生成的Stable Diffusion XL微调:字幕规模的比较

Fine-Tuning Stable Diffusion XL for Stylistic Icon Generation: A Comparison of Caption Size

Georgia Institute of Technology(佐治亚理工学院)

Youssef Sultan, Jiangqin Ma, Yu-Ying Liao

AI总结:

本研究针对商业2D风格图标生成场景,探索Stable Diffusion XL的不同微调方法,分析FID与CLIP评分的局限性,提出需专用评估指标与微调方案以提升商业图标生成质量。

摘要:

本文展示了Stable Diffusion XL的不同微调方法,包括推理步数调整,以及为每张图像定制字幕以匹配商业2D图标训练集风格的图像生成需求。我们还阐明了准确定义“高质量”的重要性,尤其是在商业使用环境中。随着生成式AI模型持续获得广泛认可与应用,出现了多种针对不同应用场景优化和评估这类模型的方法。具体而言,Stable Diffusion XL和DALL-E 3这类文本生成图像模型需要独特的评估方案,才能根据特定风格有效生成高质量图标。尽管部分基于特定风格生成的图像可能拥有更低的FID分数(即表现更优),但我们证明,即便对于栅格化图标,这一点本身也并非绝对。FID分数反映的是生成图像与整体训练集的相似性,而CLIP分数衡量的是生成图像与其文本描述的对齐程度。我们发现,FID分数会遗漏重要维度,比如图标中最关键的少数像素差异,而CLIP分数则会导致对图标质量的误判。CLIP模型对“相似性”的理解由其自身的训练数据塑造,并未考虑我们所选风格中的特征变化。我们的研究结果凸显了在生成高质量商业图标时,需要专门的评估指标和微调方法,这有望推动文本生成图像模型在专业设计场景中实现更有效、更具针对性的应用。

英文摘要:

In this paper, we show different fine-tuning methods for Stable Diffusion XL; this includes inference steps, and caption customization for each image to align with generating images in the style of a commercial 2D icon training set. We also show how important it is to properly define what "high-quality" really is especially for a commercial-use environment. As generative AI models continue to gain widespread acceptance and usage, there emerge many different ways to optimize and evaluate them for various applications. Specifically text-to-image models, such as Stable Diffusion XL and DALL-E 3 require distinct evaluation practices to effectively generate high-quality icons according to a specific style. Although some images that are generated based on a certain style may have a lower FID score (better), we show how this is not absolute in and of itself even for rasterized icons. While FID scores reflect the similarity of generated images to the overall training set, CLIP scores measure the alignment between generated images and their textual descriptions. We show how FID scores miss significant aspects, such as the minority of pixel differences that matter most in an icon, while CLIP scores result in misjudging the quality of icons. The CLIP model's understanding of "similarity" is shaped by its own training data; which does not account for feature variation in our style of choice. Our findings highlight the need for specialized evaluation metrics and fine-tuning approaches when generating high-quality commercial icons, potentially leading to more effective and tailored applications of text-to-image models in professional design contexts.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑