Culture-TRIP：基于迭代提示词优化的文化感知文本生成图像方法
Culture-TRIP: Culturally-Aware Text-to-Image Generation with Iterative Prompt Refinement
AI总结：
针对文本生成图像模型对西方文化中代表性不足的文化概念生成效果差的问题，提出Culture-TRIP方法，通过检索文化信息、结合大模型迭代优化提示词，经多国用户调研验证可提升图像与文化类提示词的对齐度。
摘要：
包括Stable Diffusion在内的文本生成图像模型，在生成与给定提示词语义高度对齐的图像方面已取得显著进展。然而，对于西方文化中知名度较低或代表性不足的文化概念或物品（例如韩国器具“hangari”），现有模型可能无法生成合适的图像。本文提出一种新方法——基于迭代提示词优化的文化感知文本生成图像方法（Culture-TRIP），该方法通过优化提示词，提升文本生成图像模型输出图像与文本中此类文化名词的对齐度。我们的方法包括两个步骤：(1) 检索提示词中文化名词相关的文化背景与视觉细节；(2) 基于一套文化标准与大语言模型，迭代优化并评估提示词。优化过程利用从Wikipedia和网络检索到的信息。我们对来自8个国家的66名参与者开展了用户调研，结果表明所提方法增强了图像与提示词的对齐度。尤其是C-TRIP，在生成图像与代表性不足的文化名词的对齐度上表现出提升。相关资源可访问https://shane3606.github.io/Culture-TRIP获取。
英文摘要：
Text-to-Image models, including Stable Diffusion, have significantly improved in generating images that are highly semantically aligned with the given prompts. However, existing models may fail to produce appropriate images for the cultural concepts or objects that are not well known or underrepresented in western cultures, such as `hangari' (Korean utensil). In this paper, we propose a novel approach, Culturally-Aware Text-to-Image Generation with Iterative Prompt Refinement (Culture-TRIP), which refines the prompt in order to improve the alignment of the image with such culture nouns in text-to-image models. Our approach (1) retrieves cultural contexts and visual details related to the culture nouns in the prompt and (2) iteratively refines and evaluates the prompt based on a set of cultural criteria and large language models. The refinement process utilizes the information retrieved from Wikipedia and the Web. Our user survey, conducted with 66 participants from eight different countries demonstrates that our proposed approach enhances the alignment between the images and the prompts. In particular, C-TRIP demonstrates improved alignment between the generated images and underrepresented culture nouns. Resource can be found at https://shane3606.github.io/Culture-TRIP.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
