不偷工减料：视频生成中流内提示切换的状态接地转换
No Corners Cut: State-Grounded Transitions for Mid-Stream Prompt Switches in Video Generation
浏览论文内容
中文总结 AI 辅助
提出SEGUE框架，通过显式规划过渡提示和SPANDMD训练方法，在视频生成流内提示切换中实现忠实的状态转换，并在多个基准上取得最优性能。
中文摘要 AI 辅助
流式视频生成器允许用户通过流内提示切换动态调制视频合成。现有的流式方法能够响应更新的指令，但仍然会偷工减料，过早地实现目标或采取启发式捷径，绕过实现合理转换所需的必要中间状态变化。在本研究中，我们提出了SEGUE，一个新颖的框架，使这一过程显式化，并训练生成器忠实地执行这些转换。在每次切换时，一个无需训练的规划器解析最新帧和提示，编写一些具有角色和持续时间的过渡提示，然后将控制权交还给用户的提示。此外，为了解决在短时时间表上训练因果模型而不破坏预备监督的固有困难，我们引入了SPANDMD，它使用完整展开作为时间上下文来评估每个活动提示，同时仅在其分配的跨度内保留其DMD残差。在OpenTrans-360上，一个包含1,800次切换的基准，评分旧状态如何退出和新状态如何开始，SEGUE在所有八个转换指标上排名第一，并将总体得分从最强基线的0.866提高到0.887。它还在StreamAV-Bench的六个指令响应指标中的四个上排名第一，而规划器无需重新训练即可转移到冻结的自回归生成器。
英文摘要
Streaming video generators allow users to dynamically modulate video synthesis via mid-stream prompt switching. Existing streaming methods can respond to the updated instruction while still cutting corners, prematurely realizing goals or taking heuristic shortcuts that bypass necessary intermediate state changes needed for a plausible transition. In this study, we present SEGUE, a novel framework that makes this process explicit and trains the generator to execute these transitions faithfully. At each switch, a training-free planner parses the latest frame and prompts, writes a few segue prompts with roles and durations, and then hands control back to the user's prompt. Furthermore, to address the inherent difficulty of training causal models on short-lived temporal schedules without corrupting preparatory supervision, we introduce SPANDMD, which evaluates each active prompt using the full rollout as temporal context while retaining its DMD residual only within the prompt's assigned span. On OpenTrans-360, a benchmark of 1,800 switches that scores how the old state exits and the new one begins, SEGUE ranks first on all eight transition metrics and raises the overall score over the strongest baseline from 0.866 to 0.887. It also ranks first on four of six instruction-response metrics of StreamAV-Bench, while the planner transfers to frozen autoregressive generators without retraining. Project Page: https://anonymous.4open.science/w/No-Corners-Cut-6C5D/
发表机构
- University of Chinese Academy of Sciences（中国科学院大学）
- University of Science and Technology Beijing（北京科技大学）
- Kling AI
机构由 AI 辅助整理，请以论文原文为准。
