TempAct: 通过规划器-执行器强化学习推进自回归视频生成中的时间合理性
TempAct: Advancing Temporal Plausibility in Autoregressive Video Generation via Planner-Executor RL
AI总结:
提出TempAct框架,利用规划器-执行器强化学习联合优化时间分解与步骤条件执行,解决自回归视频生成中时间指令模糊、延迟反应和错误传播问题,提升时间一致性。
摘要:
自回归(AR)视频扩散模型通过缓存视觉上下文逐块合成视频,实现低延迟流式生成,但这种逐块形式使得时间指令遵循变得模糊。单个全局提示未指定每个块应实现哪个子事件,而简单切换到逐步提示往往导致延迟反应、步骤语义混合以及提示转换间的错误传播。这些失败难以通过监督微调或蒸馏单独解决:SFT存在曝光偏差,而基于rollout的蒸馏仍优化低级去噪或教师分布匹配,而非直接强制动作排序和提示转换正确性。我们通过TempAct应对这些挑战,这是一个规划器-执行器强化学习框架,联合优化时间分解和步骤条件执行,以实现时间合理的AR视频生成。TempAct使用LLM规划器探索可由视频模型执行的跨度感知步骤提示,并训练AR扩散执行器在其自身生成的历史下遵循这些提示。其关键机制是层次化群体探索:候选计划形成规划组,每个计划从共享视觉上下文诱导出多个延续的执行组,从而实现对长期时间结果的计划级信用分配和对提示切换行为的执行器级信用分配。我们进一步设计层次化奖励,结合计划质量和全视频时间反馈给规划器,以及局部转换级步骤遵循奖励、美学正则化和KL约束给执行器。在Self-Forcing和LongLive上的实验表明,TempAct在保持整体视觉质量的同时提高了时间一致性。
英文摘要:
Autoregressive (AR) video diffusion models enable low-latency streaming generation by synthesizing videos chunk by chunk with cached visual context, but this chunk-wise formulation makes temporal instruction following ambiguous. A single global prompt does not specify which sub-event should be realized in each chunk, while naively switching to step-wise prompts often leads to delayed reactions, blended step semantics, and error propagation across prompt transitions. These failures are difficult to address with supervised fine-tuning or distillation alone: SFT suffers from exposure bias, while rollout-based distillation still optimizes low-level denoising or teacher-distribution matching rather than directly enforcing action ordering and prompt-transition correctness. We address these challenges with TempAct, a planner--executor reinforcement learning framework that jointly optimizes temporal decomposition and step-conditioned execution for temporally plausible AR video generation. TempAct uses an LLM planner to explore span-aware step prompts that are executable by the video model, and trains an AR diffusion executor to follow these prompts under its own generated histories. Its key mechanism is hierarchical group exploration: candidate plans form planning groups, and each plan induces an execution group of multiple continuations from a shared visual context, enabling plan-level credit assignment for long-horizon temporal outcomes and executor-level credit assignment for prompt-switch behavior. We further design hierarchical rewards that combine plan-quality and full-video temporal feedback for the planner with local transition-level step-following rewards, aesthetic regularization, and KL constraints for the executor. Experiments on Self-Forcing and LongLive show that TempAct improves temporal consistency while preserving overall visual quality.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。