ReCap：轻量级指代 grounding 以实现连贯故事可视化
ReCap: Lightweight Referential Grounding for Coherent Story Visualization
AI总结：
ReCap 提出一种轻量级框架，通过视觉锚点提升角色稳定性与视觉一致性，无需修改基础扩散模型，在两个故事可视化基准上取得新高。
摘要：
故事可视化旨在生成一系列图像，以忠实描绘文本叙述，保持角色身份、空间配置和风格一致性。传统上依赖显式记忆库、架构扩展或辅助语言模型，导致参数增长和推理开销。我们引入ReCap，一种轻量一致性框架，通过CORE模块将代词作为视觉锚点，仅在角色被代词指代时激活，并基于前一帧进行条件传播。该设计避免了无条件跨帧条件化，仅引入149K额外参数。为进一步稳定身份，我们整合SemDrift（指导语义漂移校正），仅在训练期间应用。当文本模糊或指代时，去噪器缺乏定义身份的视觉锚点，导致角色外观漂移，SemDrift通过对齐去噪器表示与预训练DINOv3视觉嵌入，强制在零推理开销下保持语义身份稳定性。ReCap在FlintstonesSV和PororoSV两个基准上分别比先前最佳的StoryGPT-V提高2.63%和5.65%的Character-Accuracy，建立了新的最佳状态。此外，我们扩展故事可视化到源自真实电影的人本叙事，展示了ReCap在非风格化卡通领域之外的能力。
英文摘要：
Story Visualization aims to generate a sequence of images that faithfully depicts a textual narrative that preserve character identity, spatial configuration, and stylistic coherence as the narratives unfold. Maintaining such cross-frame consistency has traditionally relied on explicit memory banks, architectural expansion, or auxiliary language models, resulting in substantial parameter growth and inference overhead. We introduce ReCap, a lightweight consistency framework that improves character stability and visual fidelity without modifying the base diffusion backbone. ReCap's CORE (COnditional frame REferencing) module treats anaphors, in our case pronouns, as visual anchors, activating only when characters are referred to by a pronoun and conditioning on the preceding frame to propagate visual identity. This selective design avoids unconditional cross-frame conditioning and introduces only 149K additional parameters, a fraction of the cost of memory-bank and LLM-augmented approaches. To further stabilize identity, we incorporate SemDrift (Guided Semantic Drift Correction) applied only during training. When text is vague or referential, the denoiser lacks a visual anchor for identity-defining attributes, causing character appearance to drift across frames, SemDrift corrects this by aligning denoiser representations with pretrained DINOv3 visual embeddings, enforcing semantic identity stability at zero inference cost. ReCap outperforms previous state-of-the-art, StoryGPT-V, on the two main benchmarks for story visualization by 2.63% Character-Accuracy on FlintstonesSV and by 5.65% on PororoSV, establishing a new state-of-the-art character consistency on both benchmarks. Furthermore, we extend story visualization to human-centric narratives derived from real films, demonstrating the capability of ReCap beyond stylized cartoon domains.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
