利用视觉前缀调优与对比学习的上下文感知视觉故事讲述
Context-aware Visual Storytelling with Visual Prefix Tuning and Contrastive Learning
AI总结：
针对图像序列故事生成中的上下文与视觉变化挑战，提出基于预训练基础模型的轻量视觉-语言映射框架，并结合上下文建模与多模态对比学习，提升故事连贯性、视觉相关性和信息量。
摘要：
视觉故事讲述系统根据图像序列生成多句故事。在此任务中，捕捉上下文信息并衔接视觉变化带来了额外挑战。我们提出一个简单而有效的框架，利用预训练基础模型的泛化能力，仅训练一个轻量级视觉-语言映射网络来连接两种模态，同时融入上下文以增强连贯性。我们引入了多模态对比目标，该目标还提升了视觉相关性和故事信息量。涵盖自动指标与人工评估的大量实验结果表明，我们框架生成的故事具有多样性、连贯性、信息量和趣味性。
英文摘要：
Visual storytelling systems generate multi-sentence stories from image sequences. In this task, capturing contextual information and bridging visual variation bring additional challenges. We propose a simple yet effective framework that leverages the generalization capabilities of pretrained foundation models, only training a lightweight vision-language mapping network to connect modalities, while incorporating context to enhance coherence. We introduce a multimodal contrastive objective that also improves visual relevance and story informativeness. Extensive experimental results, across both automatic metrics and human evaluations, demonstrate that the stories generated by our framework are diverse, coherent, informative, and interesting.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
