arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 1807.08077cs.CL
收藏

创造性视觉故事的管道

A Pipeline for Creative Visual Storytelling

U.S. Army Research Laboratory(美国陆军研究实验室)

Stephanie M. Lukin, Reginald Hobbs, Clare R. Voss

摘要:

计算视觉叙事产生对事件的文本描述和在图像序列中描绘的解释。这些文本是通过自然语言处理、生成和计算机视觉的进步和跨学科方法而成为可能的。我们定义一个计算创造性的视觉故事是一个有能力改变一个故事的讲述沿着三个方面:谈论不同的环境,根据叙事目标产生变化,并适应观众的叙事。创造性叙事的这些方面及其对叙事的影响还有待于在视觉叙事中进行探索。本文提出了一个管道的任务模块,对象识别,单图像推理,多图像叙事,作为一个初步设计,建立一个创造性的视觉讲故事。我们已经在一个注释任务中对一系列图像进行了这种设计。我们提出并分析收集的语料库,并描述自动化的计划。

英文摘要:

Computational visual storytelling produces a textual description of events and interpretations depicted in a sequence of images. These texts are made possible by advances and cross-disciplinary approaches in natural language processing, generation, and computer vision. We define a computational creative visual storytelling as one with the ability to alter the telling of a story along three aspects: to speak about different environments, to produce variations based on narrative goals, and to adapt the narrative to the audience. These aspects of creative storytelling and their effect on the narrative have yet to be explored in visual storytelling. This paper presents a pipeline of task-modules, Object Identification, Single-Image Inferencing, and Multi-Image Narration, that serve as a preliminary design for building a creative visual storyteller. We have piloted this design for a sequence of images in an annotation task. We present and analyze the collected corpus and describe plans towards automation.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑