对话导演：弥合多模态叙事中对话可视化的差距
Dialogue Director: Bridging the Gap in Dialogue Visualization for Multimodal Storytelling
AI总结：
本文提出对话可视化任务及无需训练的对话导演框架，通过思维链推理、检索增强生成和多视角合成提升对话剧本到多视角分镜的转化质量，实验表明其优于现有方法。
摘要：
近年来，AI驱动的叙事技术取得了显著进展，增强了视频生成和故事可视化的能力。然而，将以对话为中心的剧本转化为连贯的分镜脚本仍然是一个重大挑战，原因在于剧本细节有限、对物理情境的理解不足，以及整合电影原则的复杂性。为了解决这些挑战，我们提出了对话可视化（Dialogue Visualization）这一新任务，旨在将对话剧本转化为动态的多视角分镜脚本。我们引入了对话导演（Dialogue Director），这是一个无需训练的多模态框架，由剧本导演（Script Director）、摄影师（Cinematographer）和分镜制作器（Storyboard Maker）组成。该框架利用大型多模态模型和基于扩散的架构，采用思维链推理（Chain-of-Thought reasoning）、检索增强生成（Retrieval-Augmented Generation）和多视角合成等技术，以提升剧本理解、物理情境理解以及电影知识的整合能力。实验结果表明，对话导演在剧本解读、物理世界理解和电影原则应用方面均优于最先进的方法，显著提升了基于对话的故事可视化的质量和可控性。
英文摘要：
Recent advances in AI-driven storytelling have enhanced video generation and story visualization. However, translating dialogue-centric scripts into coherent storyboards remains a significant challenge due to limited script detail, inadequate physical context understanding, and the complexity of integrating cinematic principles. To address these challenges, we propose Dialogue Visualization, a novel task that transforms dialogue scripts into dynamic, multi-view storyboards. We introduce Dialogue Director, a training-free multimodal framework comprising a Script Director, Cinematographer, and Storyboard Maker. This framework leverages large multimodal models and diffusion-based architectures, employing techniques such as Chain-of-Thought reasoning, Retrieval-Augmented Generation, and multi-view synthesis to improve script understanding, physical context comprehension, and cinematic knowledge integration. Experimental results demonstrate that Dialogue Director outperforms state-of-the-art methods in script interpretation, physical world understanding, and cinematic principle application, significantly advancing the quality and controllability of dialogue-based story visualization.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
