演进式叙事：使用扩散模型进行新角色定制的基准与方法
Evolving Storytelling: Benchmarks and Methods for New Character Customization with Diffusion Models
AI总结：
针对扩散模型在少样本视觉叙事中难以融入并保持新角色一致性的问题，本文构建 NewEpisode 基准并提出 EpicEvo，通过对抗性角色对齐与知识蒸馏实现单故事新角色定制。
摘要：
用于故事可视化的基于扩散的模型已在为叙事任务生成内容连贯图像方面展现出前景。然而，如何在将新角色有效融入现有叙事的同时保持角色一致性，仍是一个尚未解决的问题，尤其是在数据有限的情况下。两大局限阻碍了进展：（1）由于潜在的角色泄漏和不一致的文本标注，缺乏合适的基准；（2）区分新角色与旧角色存在挑战，导致结果模糊。为应对这些挑战，我们提出了 NewEpisode 基准，该基准包含经过精炼的数据集，旨在评估生成模型仅凭一个示例故事生成带有全新角色的新故事时的适应能力。精炼后的数据集包含改进的文本提示，并消除了角色泄漏。此外，为缓解生成结果中的角色混淆，我们提出了 EpicEvo，该方法利用一个包含新角色的故事来定制基于扩散的视觉故事生成模型，将这些角色无缝整合到既定的角色互动关系中。EpicEvo 引入了一种新颖的对抗性角色对齐模块，在扩散过程中利用新角色的示例图像逐步对齐生成图像，同时应用知识蒸馏以防止遗忘角色和背景细节。我们的评估定量表明，EpicEvo 在 NewEpisode 基准上优于现有基线，定性研究也证实其在扩散模型的视觉故事生成定制方面表现更出色。总之，EpicEvo 提供了一种仅使用一个示例故事即可纳入新角色的有效方式，为连载漫画等应用开启了新的可能性。
英文摘要：
Diffusion-based models for story visualization have shown promise in generating content-coherent images for storytelling tasks. However, how to effectively integrate new characters into existing narratives while maintaining character consistency remains an open problem, particularly with limited data. Two major limitations hinder the progress: (1) the absence of a suitable benchmark due to potential character leakage and inconsistent text labeling, and (2) the challenge of distinguishing between new and old characters, leading to ambiguous results. To address these challenges, we introduce the NewEpisode benchmark, comprising refined datasets designed to evaluate generative models' adaptability in generating new stories with fresh characters using just a single example story. The refined dataset involves refined text prompts and eliminates character leakage. Additionally, to mitigate the character confusion of generated results, we propose EpicEvo, a method that customizes a diffusion-based visual story generation model with a single story featuring the new characters seamlessly integrating them into established character dynamics. EpicEvo introduces a novel adversarial character alignment module to align the generated images progressively in the diffusive process, with exemplar images of new characters, while applying knowledge distillation to prevent forgetting of characters and background details. Our evaluation quantitatively demonstrates that EpicEvo outperforms existing baselines on the NewEpisode benchmark, and qualitative studies confirm its superior customization of visual story generation in diffusion models. In summary, EpicEvo provides an effective way to incorporate new characters using only one example story, unlocking new possibilities for applications such as serialized cartoons.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
