StoryMem: 基于记忆的多镜头长视频叙事
StoryMem: Multi-shot Long Video Storytelling with Memory
AI总结：
StoryMem通过基于记忆的迭代镜头合成，实现了高质量多镜头视频叙事，提升了跨镜头一致性与审美质量。
摘要：
视觉叙事需要生成具有电影质感和长距离一致性的多镜头视频。受人类记忆启发，我们提出StoryMem，一种将长格式视频叙事重新表述为迭代镜头合成的范式，该范式基于显式的视觉记忆进行条件生成，将预训练的单镜头视频扩散模型转化为多镜头叙事者。这通过一种新颖的内存到视频（M2V）设计实现，该设计维护了一个紧凑且动态更新的关键帧内存库，包含来自历史生成镜头的关键帧。存储的内存通过潜在拼接和负RoPE位移注入到单镜头视频扩散模型中，仅需LoRA微调。此外，语义关键帧选择策略与审美偏好过滤进一步确保了生成过程中的信息性和稳定性。此外，所提出的框架自然支持流畅的镜头过渡和定制化的叙事生成应用。为了便于评估，我们引入了ST-Bench，一个多样化的多镜头视频叙事基准。大量实验表明，StoryMem在保持高审美质量和提示符一致性的同时，实现了优于先前方法的跨镜头一致性，标志着向连贯的分钟级视频叙事迈出重要一步。
英文摘要：
Visual storytelling requires generating multi-shot videos with cinematic quality and long-range consistency. Inspired by human memory, we propose StoryMem, a paradigm that reformulates long-form video storytelling as iterative shot synthesis conditioned on explicit visual memory, transforming pre-trained single-shot video diffusion models into multi-shot storytellers. This is achieved by a novel Memory-to-Video (M2V) design, which maintains a compact and dynamically updated memory bank of keyframes from historical generated shots. The stored memory is then injected into single-shot video diffusion models via latent concatenation and negative RoPE shifts with only LoRA fine-tuning. A semantic keyframe selection strategy, together with aesthetic preference filtering, further ensures informative and stable memory throughout generation. Moreover, the proposed framework naturally accommodates smooth shot transitions and customized story generation applications. To facilitate evaluation, we introduce ST-Bench, a diverse benchmark for multi-shot video storytelling. Extensive experiments demonstrate that StoryMem achieves superior cross-shot consistency over previous methods while preserving high aesthetic quality and prompt adherence, marking a significant step toward coherent minute-long video storytelling.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
