arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2404.08799cs.CVcs.AIcs.HCcs.LG
收藏

用于量化扩散模型图像生成一致性的语义方法

Semantic Approach to Quantifying the Consistency of Diffusion Model Image Generation

Duke University(杜克大学)

Brinnae Bent

AI总结:

本研究提出基于成对平均CLIP得分的语义一致性评分,量化扩散模型图像生成一致性,验证了其与人工标注94%的一致率,可助力模型评估与选型。

摘要:

本研究明确了扩散模型图像生成的可重复性(即一致性)需要一种可解释的量化评分指标。我们提出了一种语义方法,采用成对平均CLIP(对比语言-图像预训练)得分作为我们的语义一致性评分。我们将该指标用于比较两款最先进的开源图像生成扩散模型Stable Diffusion XL与PixArt-α,发现两款模型的语义一致性得分存在统计学意义上的显著差异。语义一致性评分选出的模型与汇总的人工标注结果的一致率为94%。我们还探究了SDXL及其经LoRA微调版本的一致性,发现微调后模型生成图像的语义一致性显著更高。本文提出的语义一致性评分提供了一种图像生成对齐度的衡量标准,可助力针对特定任务评估模型架构,并为模型选择相关的知情决策提供支持。

英文摘要:

In this study, we identify the need for an interpretable, quantitative score of the repeatability, or consistency, of image generation in diffusion models. We propose a semantic approach, using a pairwise mean CLIP (Contrastive Language-Image Pretraining) score as our semantic consistency score. We applied this metric to compare two state-of-the-art open-source image generation diffusion models, Stable Diffusion XL and PixArt-α, and we found statistically significant differences between the semantic consistency scores for the models. Agreement between the Semantic Consistency Score selected model and aggregated human annotations was 94%. We also explored the consistency of SDXL and a LoRA-fine-tuned version of SDXL and found that the fine-tuned model had significantly higher semantic consistency in generated images. The Semantic Consistency Score proposed here offers a measure of image generation alignment, facilitating the evaluation of model architectures for specific tasks and aiding in informed decision-making regarding model selection.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑