关注时间：时间可控的多事件视频生成
Mind the Time: Temporally-Controlled Multi-Event Video Generation
AI总结：
本文提出 MinT，通过将每个事件绑定到特定时间段并设计 ReRoPE 时间位姿编码，首次实现多事件视频生成的精确时间控制，实验表明其性能大幅超越现有模型。
摘要：
真实世界视频由一系列事件组成。现有视频生成器依赖单段文本作为输入，无法生成具有精确时间控制的事件序列。当要求用单一提示描述多个事件时，这类方法常常忽略部分事件或无法按正确顺序安排它们。为解决这一局限，我们提出 MinT，一个具有时间控制能力的多事件视频生成器。我们的核心思想是将每个事件绑定到生成视频中的特定时间段，使模型能够一次专注于一个事件。为了实现事件描述与视频 token 之间的时间感知交互，我们设计了一种基于时间的位姿编码方法，称为 ReRoPE。该编码有助于引导交叉注意力操作。通过在具有时间锚定数据上微调预训练视频扩散 transformer，我们的方法生成连贯视频，事件之间平滑衔接。在文献中，我们的模型首次实现了对生成视频中事件时序的控制。大量实验表明，MinT 大幅优于现有商业和开源模型。
英文摘要：
Real-world videos consist of sequences of events. Generating such sequences with precise temporal control is infeasible with existing video generators that rely on a single paragraph of text as input. When tasked with generating multiple events described using a single prompt, such methods often ignore some of the events or fail to arrange them in the correct order. To address this limitation, we present MinT, a multi-event video generator with temporal control. Our key insight is to bind each event to a specific period in the generated video, which allows the model to focus on one event at a time. To enable time-aware interactions between event captions and video tokens, we design a time-based positional encoding method, dubbed ReRoPE. This encoding helps to guide the cross-attention operation. By fine-tuning a pre-trained video diffusion transformer on temporally grounded data, our approach produces coherent videos with smoothly connected events. For the first time in the literature, our model offers control over the timing of events in generated videos. Extensive experiments demonstrate that MinT outperforms existing commercial and open-source models by a large margin.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
