通过推理时间引导进行组合视频生成
Compositional Video Generation via Inference-Time Guidance
AI总结：
本文提出CVG方法，通过利用交叉注意力图中的空间时间接地信号，提升冻结文本到视频模型的组合忠实度，无需修改模型结构或微调生成器。
摘要：
文本到视频扩散模型能够生成逼真的视频，但往往在需要细粒度组合理解的提示上表现不佳，例如实体之间的关系、属性、动作和运动方向。我们假设这些失败不需要通过重新训练生成器来解决，而是可以通过利用模型自身的内部接地信号来引导去噪过程来缓解。我们提出了CVG，一种用于改进冻结文本到视频模型中组合忠实度的推理时间引导方法。我们的关键观察是交叉注意力图已经编码了提示概念如何在空间和时间上接地。我们训练了一个轻量级的组合分类器来利用这些注意力特征，并在去噪的早期步骤中使用其梯度来引导潜在轨迹向期望的组合方向发展。基于冻结的VLM主干网络，分类器在语义相关的组合标签上进行转移，而不是仅依赖于狭窄的类别特定特征。CVG在不修改模型架构、微调生成器或需要布局、框或其他用户提供的控制的情况下提高了组合生成能力。在组合文本到视频基准测试中的实验表明，提高了提示的忠实度，同时保持了底层生成器的视觉质量。
英文摘要：
Text-to-video diffusion models generate realistic videos, but often fail on prompts requiring fine-grained compositional understanding, such as relations between entities, attributes, actions, and motion directions. We hypothesize that these failures need not be addressed by retraining the generator, but can instead be mitigated by steering the denoising process using the model's own internal grounding signals. We propose \textbf{CVG}, an inference-time guidance method for improving compositional faithfulness in frozen text-to-video models. Our key observation is that cross-attention maps already encode how prompt concepts are grounded across space and time. We train a lightweight compositional classifier on these attention features and use its gradients during early denoising steps to steer the latent trajectory toward the desired composition. Built on a frozen VLM backbone, the classifier transfers across semantically related composition labels rather than relying only on narrow category-specific features. CVG improves compositional generation without modifying the model architecture, fine-tuning the generator, or requiring layouts, boxes, or other user-supplied controls. Experiments on compositional text-to-video benchmarks show improved prompt faithfulness while preserving the visual quality of the underlying generator.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
