多语言DALL-E故事时间
Multi-Lingual DALL-E Storytime
摘要:
虽然人工智能(AI)语言模型的最新进展在处理英语文本时表现出了最先进的性能,但在其他语言中不存在等效模型,或者没有达到相同的性能水平。人工智能进步的这种不利影响增加了世界各地不同人群获得新技术之间的差距。这种不经意的偏见主要歧视那些英语技能不太发达的人,例如,非英语国家的儿童。随着近年来人工智能研究的重大进展,OpenAI最近推出了DALL-E:一个基于英文文本提示创建图像的强大工具。虽然DALL-E是许多应用程序的一个有前途的工具,但当以不同语言输入时,其性能下降,限制了其受众并加深了人群之间的差距。当前DALL-E模型的另一个限制是,它只允许响应于给定的输入提示创建几个图像,而不是一系列连续的连贯帧,这些帧讲述一个故事或描述一个随时间变化的过程。在这里,我们提出了一个易于使用的自动DALL-E讲故事的框架,利用现有的DALL-E模型,使非英语歌曲和故事的快速和连贯的可视化,推动DALL-E目前提供的一步一步的选项的限制。我们表明,我们的框架是能够有效地可视化的故事,从非英语文本和描绘的情节随着时间的推移的变化。它还能够创建一个叙事,并在跨框架的描述中保持可解释的变化。此外,我们的框架为用户提供了对故事元素指定约束的能力,例如特定的位置或上下文,并在整个可视化过程中保持一致的风格。
英文摘要:
While recent advancements in artificial intelligence (AI) language models demonstrate cutting-edge performance when working with English texts, equivalent models do not exist in other languages or do not reach the same performance level. This undesired effect of AI advancements increases the gap between access to new technology from different populations across the world. This unsought bias mainly discriminates against individuals whose English skills are less developed, e.g., non-English speakers children. Following significant advancements in AI research in recent years, OpenAI has recently presented DALL-E: a powerful tool for creating images based on English text prompts. While DALL-E is a promising tool for many applications, its decreased performance when given input in a different language, limits its audience and deepens the gap between populations. An additional limitation of the current DALL-E model is that it only allows for the creation of a few images in response to a given input prompt, rather than a series of consecutive coherent frames that tell a story or describe a process that changes over time. Here, we present an easy-to-use automatic DALL-E storytelling framework that leverages the existing DALL-E model to enable fast and coherent visualizations of non-English songs and stories, pushing the limit of the one-step-at-a-time option DALL-E currently offers. We show that our framework is able to effectively visualize stories from non-English texts and portray the changes in the plot over time. It is also able to create a narrative and maintain interpretable changes in the description across frames. Additionally, our framework offers users the ability to specify constraints on the story elements, such as a specific location or context, and to maintain a consistent style throughout the visualization.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。