arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2211.07441cs.CLcs.CVcs.LG
收藏

Multi-VQG:为多个图像生成吸引人的问题

Multi-VQG: Generating Engaging Questions for Multiple Images

University of Massachusetts Amherst(马萨诸塞大学阿默斯特分校);University of Illinois Urbana-Champaign(伊利诺伊大学厄巴纳-香槟分校);Pennsylvania State University(宾夕法尼亚州立大学);Institute of Information Science, Academia Sinica(中央研究院信息科学研究所)

Min-Hsuan Yeh, Vicent Chen, Ting-Hao 'Kenneth' Haung, Lun-Wei Ku

摘要:

生成引人入胜的内容最近在NLP社区引起了广泛的关注。提问是回应照片和提高意识的自然方式。然而,在传统的问答(QA)数据集中,大多数问题的答案是事实,这降低了个人的回答意愿。此外,传统的可视化问题生成(VQG)限制了用于问题生成的源数据到单个图像,导致理解底层事件的时间序列信息的能力有限。在本文中,我们提出了从多个图像生成引人入胜的问题。我们提出了MVQG,一个新的数据集,并建立了一系列的基线,包括端到端和双阶段架构。结果表明,在图像序列背后构建故事使模型能够生成引人入胜的问题,这证实了我们的假设,即人们通常在提出问题之前在脑海中构建事件的图片。这些结果为视觉和语言模型提出了一个令人兴奋的挑战,即隐式地构建一系列照片背后的故事,以允许创造力和经验共享,从而吸引下游应用程序的注意力。

英文摘要:

Generating engaging content has drawn much recent attention in the NLP community. Asking questions is a natural way to respond to photos and promote awareness. However, most answers to questions in traditional question-answering (QA) datasets are factoids, which reduce individuals' willingness to answer. Furthermore, traditional visual question generation (VQG) confines the source data for question generation to single images, resulting in a limited ability to comprehend time-series information of the underlying event. In this paper, we propose generating engaging questions from multiple images. We present MVQG, a new dataset, and establish a series of baselines, including both end-to-end and dual-stage architectures. Results show that building stories behind the image sequence enables models to generate engaging questions, which confirms our assumption that people typically construct a picture of the event in their minds before asking questions. These results open up an exciting challenge for visual-and-language models to implicitly construct a story behind a series of photos to allow for creativity and experience sharing and hence draw attention to downstream applications.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑