通过事实图像检索解决文本生成图像中的图像幻觉问题
Addressing Image Hallucination in Text-to-Image Generation through Factual Image Retrieval
AI总结：
针对文本生成图像中的图像幻觉问题，研究将其分为三类，提出借助外部检索的事实图像、搭配InstructPix2Pix或IP-Adapter编辑工具生成符合事实与常识的图像的方法。
摘要：
随着扩散模型的出现，文本生成图像已取得显著进展。但这类模型生成的图像往往存在事实不一致问题，无法准确反映输入文本提示所传达的事实信息与常识，我们将该问题称为图像幻觉。借鉴大语言模型幻觉相关研究，我们将此问题分为三类，并提出一种利用从外部来源检索到的事实图像来生成真实图像的方法。根据幻觉的类型，我们采用现成的图像编辑工具InstructPix2Pix或IP-Adapter，来利用检索图像中的事实信息。该方法能够生成准确反映事实与常识的图像。
英文摘要：
Text-to-image generation has shown remarkable progress with the emergence of diffusion models. However, these models often generate factually inconsistent images, failing to accurately reflect the factual information and common sense conveyed by the input text prompts. We refer to this issue as Image hallucination. Drawing from studies on hallucinations in language models, we classify this problem into three types and propose a methodology that uses factual images retrieved from external sources to generate realistic images. Depending on the nature of the hallucination, we employ off-the-shelf image editing tools, either InstructPix2Pix or IP-Adapter, to leverage factual information from the retrieved image. This approach enables the generation of images that accurately reflect the facts and common sense.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
