（尚）非全貌：评估视觉故事讲述需要的不只是衡量连贯性、视觉锚定和重复性
Not (yet) the whole story: Evaluating Visual Storytelling Requires More than Measuring Coherence, Grounding, and Repetition
AI总结：
本文提出从视觉锚定、连贯性和重复性衡量故事类人性的评估方法，发现LLaVA仅略优于小50倍的TAPM，且人工评估显示好故事还需更多维度。
摘要：
视觉故事讲述是指给定一个按时间顺序排列的图像序列，生成自然语言故事。这项任务不仅对模型具有挑战性，而且由于人们对于什么造就一个“好”故事尚无共识，使用自动指标进行评估也非常困难。本文提出一种新方法，从人类相似性角度，围绕先前工作强调的三个关键方面衡量故事质量：视觉锚定、连贯性和重复性。随后，我们使用该方法评估若干模型生成的故事，结果表明基础模型 LLaVA 取得最佳成绩，但相比规模小50倍的视觉故事讲述模型 TAPM 仅略有优势。升级 TAPM 的视觉组件和语言组件后所得模型，能够以相对较少的参数量获得具有竞争力的性能。最后，我们开展了一项人工评估研究，其结果表明，一个“好”故事可能需要的不只是达到类人水平的视觉锚定、连贯性和重复性。
英文摘要：
Visual storytelling consists in generating a natural language story given a temporally ordered sequence of images. This task is not only challenging for models, but also very difficult to evaluate with automatic metrics since there is no consensus about what makes a story 'good'. In this paper, we introduce a novel method that measures story quality in terms of human likeness regarding three key aspects highlighted in previous work: visual grounding, coherence, and repetitiveness. We then use this method to evaluate the stories generated by several models, showing that the foundation model LLaVA obtains the best result, but only slightly so compared to TAPM, a 50-times smaller visual storytelling model. Upgrading the visual and language components of TAPM results in a model that yields competitive performance with a relatively low number of parameters. Finally, we carry out a human evaluation study, whose results suggest that a 'good' story may require more than a human-like level of visual grounding, coherence, and repetition.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
