Multi-VQG:为多个图像生成吸引人的问题
Multi-VQG: Generating Engaging Questions for Multiple Images
摘要:
生成引人入胜的内容最近在NLP社区引起了广泛的关注。提问是回应照片和提高意识的自然方式。然而,在传统的问答(QA)数据集中,大多数问题的答案是事实,这降低了个人的回答意愿。此外,传统的可视化问题生成(VQG)限制了用于问题生成的源数据到单个图像,导致理解底层事件的时间序列信息的能力有限。在本文中,我们提出了从多个图像生成引人入胜的问题。我们提出了MVQG,一个新的数据集,并建立了一系列的基线,包括端到端和双阶段架构。结果表明,在图像序列背后构建故事使模型能够生成引人入胜的问题,这证实了我们的假设,即人们通常在提出问题之前在脑海中构建事件的图片。这些结果为视觉和语言模型提出了一个令人兴奋的挑战,即隐式地构建一系列照片背后的故事,以允许创造力和经验共享,从而吸引下游应用程序的注意力。
英文摘要:
Generating engaging content has drawn much recent attention in the NLP community. Asking questions is a natural way to respond to photos and promote awareness. However, most answers to questions in traditional question-answering (QA) datasets are factoids, which reduce individuals' willingness to answer. Furthermore, traditional visual question generation (VQG) confines the source data for question generation to single images, resulting in a limited ability to comprehend time-series information of the underlying event. In this paper, we propose generating engaging questions from multiple images. We present MVQG, a new dataset, and establish a series of baselines, including both end-to-end and dual-stage architectures. Results show that building stories behind the image sequence enables models to generate engaging questions, which confirms our assumption that people typically construct a picture of the event in their minds before asking questions. These results open up an exciting challenge for visual-and-language models to implicitly construct a story behind a series of photos to allow for creativity and experience sharing and hence draw attention to downstream applications.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。