InteractiveVideo：基于协同多模态指令的用户中心可控视频生成
InteractiveVideo: User-Centric Controllable Video Generation with Synergistic Multimodal Instructions
AI总结：
InteractiveVideo提出协同多模态指令机制，支持用户通过文本、图像、绘画、拖放等方式动态交互，实现视频生成的细粒度可控与迭代优化。
摘要：
我们提出了InteractiveVideo，一个以用户为中心的视频生成框架。与传统的基于用户提供的图像或文本进行操作的生成方法不同，我们的框架专为动态交互而设计，允许用户在整个生成过程中通过多种直观机制（如文本和图像提示、绘画、拖放等）来指导生成模型。我们提出了一种协同多模态指令机制，旨在将用户的多模态指令无缝集成到生成模型中，从而促进用户输入与生成过程之间的协作和响应式交互。这种方法通过精确有效的用户指令，实现对生成结果的迭代和细粒度细化。借助InteractiveVideo，用户可以灵活地精心定制视频的关键方面。他们可以绘制参考图像、编辑语义并调整视频运动，直到完全满足其需求。代码、模型和演示可在https://github.com/invictus717/InteractiveVideo获取。
英文摘要：
We introduce $\textit{InteractiveVideo}$, a user-centric framework for video generation. Different from traditional generative approaches that operate based on user-provided images or text, our framework is designed for dynamic interaction, allowing users to instruct the generative model through various intuitive mechanisms during the whole generation process, e.g. text and image prompts, painting, drag-and-drop, etc. We propose a Synergistic Multimodal Instruction mechanism, designed to seamlessly integrate users' multimodal instructions into generative models, thus facilitating a cooperative and responsive interaction between user inputs and the generative process. This approach enables iterative and fine-grained refinement of the generation result through precise and effective user instructions. With $\textit{InteractiveVideo}$, users are given the flexibility to meticulously tailor key aspects of a video. They can paint the reference image, edit semantics, and adjust video motions until their requirements are fully met. Code, models, and demo are available at https://github.com/invictus717/InteractiveVideo
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
