MagicScroll：基于多层语义感知去噪的视觉叙事非典型宽高比图像生成
MagicScroll: Nontypical Aspect-Ratio Image Generation for Visual Storytelling via Multi-Layered Semantic-Aware Denoising
AI总结：
针对视觉叙事中非典型宽高比图像生成的可控性与连贯性难题，提出多层语义感知去噪的扩散框架MagicScroll，构建首个相关基准，验证了其在文本对齐、视觉连贯等方面的优势。
摘要：
视觉叙事常使用卷轴画、连环漫画和全景图等非典型宽高比图像来打造富有表现力与吸引力的叙事效果。尽管生成式人工智能已取得巨大成功，展现出重塑创意产业的潜力，但生成具备任意尺寸且风格、概念、布局可控的连贯且引人入胜的内容仍是一项挑战，而这些特性对视觉叙事至关重要。为克服现有方法存在的内容重复、风格不一致和可控性不足等缺陷，我们提出MagicScroll——一种基于渐进式扩散的多层图像生成框架，配备新颖的语义感知去噪流程。该模型可在物体、场景和背景层级，通过文本、图像和布局条件实现对生成图像的细粒度控制。我们还构建了首个面向视觉叙事的非典型宽高比图像生成基准，涵盖绘画、漫画和电影级全景图等媒介，并配有定制化指标用于系统性评估。通过对比实验和消融实验，MagicScroll在匹配叙事文本、提升视觉连贯性和吸引受众方面展现出优异效果。我们计划发布代码和基准，以期推动人工智能研究者与视觉叙事领域创意从业者的更好协作。
英文摘要：
Visual storytelling often uses nontypical aspect-ratio images like scroll paintings, comic strips, and panoramas to create an expressive and compelling narrative. While generative AI has achieved great success and shown the potential to reshape the creative industry, it remains a challenge to generate coherent and engaging content with arbitrary size and controllable style, concept, and layout, all of which are essential for visual storytelling. To overcome the shortcomings of previous methods including repetitive content, style inconsistency, and lack of controllability, we propose MagicScroll, a multi-layered, progressive diffusion-based image generation framework with a novel semantic-aware denoising process. The model enables fine-grained control over the generated image on object, scene, and background levels with text, image, and layout conditions. We also establish the first benchmark for nontypical aspect-ratio image generation for visual storytelling including mediums like paintings, comics, and cinematic panoramas, with customized metrics for systematic evaluation. Through comparative and ablation studies, MagicScroll showcases promising results in aligning with the narrative text, improving visual coherence, and engaging the audience. We plan to release the code and benchmark in the hope of a better collaboration between AI researchers and creative practitioners involving visual storytelling.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
