StoryWeaver：用于知识增强故事角色定制的统一世界模型
StoryWeaver: A Unified World Model for Knowledge-Enhanced Story Character Customization
AI总结：
针对故事可视化中角色身份保持与文本语义对齐失衡的问题，提出基于角色图知识增强的StoryWeaver模型，通过C-CG和KE-SG实现一致且语义丰富的故事生成，在TBC-Bench上显著提升性能。
摘要：
故事可视化在人工智能领域日益受到关注。然而，现有方法在保持角色身份一致性与文本语义对齐之间的平衡方面仍面临挑战，这主要归因于对故事场景缺乏详细的语义建模。为应对这一挑战，我们提出了一种新颖的知识图谱，即角色图（Character Graph, CG），它全面表示各种与故事相关的知识，包括角色、与角色相关的属性以及角色之间的关系。随后，我们引入了StoryWeaver，一种通过角色图实现定制（Customization via Character Graph, C-CG）的图像生成器，能够实现具有丰富文本语义的一致故事可视化。为了进一步提升多角色生成性能，我们将知识增强的空间引导（Knowledge-Enhanced Spatial Guidance, KE-SG）融入StoryWeaver，以将角色语义精确注入生成过程。为验证所提方法的有效性，我们使用一个名为TBC-Bench的新基准进行了大量实验。实验证实，我们的StoryWeaver不仅在创建生动的视觉故事情节方面表现出色，而且在各种场景下准确传达角色身份方面也表现优异，且具有相当高的存储效率，例如，平均DINO-I提升+9.03%，CLIP-T提升+13.44%。此外，还进行了消融实验以验证所提模块的优越性。代码和数据集已在https://github.com/Aria-Zhangjl/StoryWeaver发布。
英文摘要：
Story visualization has gained increasing attention in artificial intelligence. However, existing methods still struggle with maintaining a balance between character identity preservation and text-semantics alignment, largely due to a lack of detailed semantic modeling of the story scene. To tackle this challenge, we propose a novel knowledge graph, namely Character Graph (\textbf{CG}), which comprehensively represents various story-related knowledge, including the characters, the attributes related to characters, and the relationship between characters. We then introduce StoryWeaver, an image generator that achieve Customization via Character Graph (\textbf{C-CG}), capable of consistent story visualization with rich text semantics. To further improve the multi-character generation performance, we incorporate knowledge-enhanced spatial guidance (\textbf{KE-SG}) into StoryWeaver to precisely inject character semantics into generation. To validate the effectiveness of our proposed method, extensive experiments are conducted using a new benchmark called TBC-Bench. The experiments confirm that our StoryWeaver excels not only in creating vivid visual story plots but also in accurately conveying character identities across various scenarios with considerable storage efficiency, \emph{e.g.}, achieving an average increase of +9.03\% DINO-I and +13.44\% CLIP-T. Furthermore, ablation experiments are conducted to verify the superiority of the proposed module. Codes and datasets are released at https://github.com/Aria-Zhangjl/StoryWeaver.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
