StoryMovie: 一个用于视觉故事语义对齐的语料库，结合电影剧本和字幕
StoryMovie: A Dataset for Semantic Alignment of Visual Stories with Movie Scripts and Subtitles
AI总结：
StoryMovie通过结合电影剧本和字幕，提升视觉叙事模型的语义对齐能力，使对话归属更准确。
摘要：
视觉叙事模型即使能够正确地将实体定位在图像中，仍可能产生语义关系的幻觉，生成错误的对话归属、角色互动或情感状态。我们介绍了StoryMovie，一个包含1,757个故事的语料库，通过LCS匹配与电影剧本和字幕对齐。我们的对齐流程同步了剧本对话与字幕时间戳，通过将剧本中的角色名称与字幕中的时间位置链接，实现对话归属。使用这种对齐内容，我们生成的故事在保持视觉定位标签的同时，还融入了真实的角色名称、对话和关系动态。我们在此数据集上微调Qwen Storyteller3，基于先前在视觉定位和实体重识别方面的研究成果。使用DeepSeek V3作为评估者，结果显示，Storyteller3在字幕对齐上以89.9%的胜率击败基础Qwen2.5-VL 7B。与未使用剧本定位的Storyteller相比，Storyteller3在字幕对齐上达到48.5%的胜率，相比38.0%，证实了语义对齐在对话归属方面比仅依赖视觉定位有所提升。
英文摘要：
Visual storytelling models that correctly ground entities in images may still hallucinate semantic relationships, generating incorrect dialogue attribution, character interactions, or emotional states. We introduce StoryMovie, a dataset of 1,757 stories aligned with movie scripts and subtitles through LCS matching. Our alignment pipeline synchronizes screenplay dialogue with subtitle timestamps, enabling dialogue attribution by linking character names from scripts to temporal positions from subtitles. Using this aligned content, we generate stories that maintain visual grounding tags while incorporating authentic character names, dialogue, and relationship dynamics. We fine-tune Qwen Storyteller3 on this dataset, building on prior work in visual grounding and entity re-identification. Evaluation using DeepSeek V3 as judge shows that Storyteller3 achieves an 89.9% win rate against base Qwen2.5-VL 7B on subtitle alignment. Compared to Storyteller, trained without script grounding, Storyteller3 achieves 48.5% versus 38.0%, confirming that semantic alignment progressively improves dialogue attribution beyond visual grounding alone.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
