arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2307.07589cs.HC
收藏

GenAssist: 让图像生成变得可及

GenAssist: Making Image Generation Accessible

Mina Huh, Yi-Hao Peng, Amy Pavel

AI总结:

针对盲人和低视力创作者难以评估文本生成图像内容与质量的问题,研究提出GenAssist系统,结合大语言模型与视觉-语言模型提供图像验证、细节查询和异同总结功能,经12名BLV创作者测试,提升了视觉创作的可及性。

摘要:

盲人和低视力(BLV)创作者使用图像与有视力的受众交流。然而,由于难以使用创作工具或评估图像搜索结果,BLV创作者在创建或检索图像时面临挑战。因此,创作者会限制自己创作的图像类型,或者招募有视力的合作者。虽然文本到图像生成模型能让创作者基于文本描述(即提示词)生成高保真图像,但创作者很难评估生成图像的内容和质量。我们提出了GenAssist,一个让文本到图像生成变得可及的系统。使用我们的界面,创作者可以验证生成的候选图像是否遵循提示词,获取图像中未在提示词里指定的额外细节,还能快速浏览候选图像之间异同点的摘要。为支撑该界面,GenAssist使用大语言模型生成视觉问题,用视觉-语言模型提取答案,再用大语言模型总结结果。我们对12名BLV创作者的研究表明,GenAssist能够实现并简化图像选择和生成的过程,让视觉创作对所有人都更具可及性。

英文摘要:

Blind and low vision (BLV) creators use images to communicate with sighted audiences. However, creating or retrieving images is challenging for BLV creators as it is difficult to use authoring tools or assess image search results. Thus, creators limit the types of images they create or recruit sighted collaborators. While text-to-image generation models let creators generate high-fidelity images based on a text description (i.e. prompt), it is difficult to assess the content and quality of generated images. We present GenAssist, a system to make text-to-image generation accessible. Using our interface, creators can verify whether generated image candidates followed the prompt, access additional details in the image not specified in the prompt, and skim a summary of similarities and differences between image candidates. To power the interface, GenAssist uses a large language model to generate visual questions, vision-language models to extract answers, and a large language model to summarize the results. Our study with 12 BLV creators demonstrated that GenAssist enables and simplifies the process of image selection and generation, making visual authoring more accessible to all.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑