因果引导扩散用于自动化视频反事实生成
Causally Steered Diffusion for Automated Video Counterfactual Generation
AI总结：
提出CSVC框架，通过将因果图编码为文本提示并利用VLM文本损失优化，引导潜在扩散模型生成因果一致的视频反事实，无需访问内部机制，实现最先进因果有效性且保持时间与视觉质量。
摘要：
将文本到图像（T2I）潜在扩散模型（LDMs）应用于视频编辑已展现出强大的视觉保真度和可控性，但在维持视频数据生成过程中固有的因果关系方面仍存在挑战。如果忽略这些关系，对因果相关属性的编辑往往会产生不真实或误导性的结果。在这项工作中，我们引入了一个因果一致的视频反事实生成框架，将其表述为一个分布外（OOD）预测问题。我们通过将因果图中指定的关系编码到文本提示中来嵌入先验因果知识，并使用基于视觉语言模型（VLM）的文本损失优化这些提示来引导生成过程。该损失鼓励LDM的潜在空间以反事实的形式捕获OOD变化，有效地将生成引导向因果上有意义的替代方案。所提出的框架名为CSVC，与底层视频编辑系统无关，不需要访问其内部机制或进行微调。我们使用标准视频质量指标和反事实特定标准（如因果有效性和最小性）来评估我们的方法。实验结果表明，CSVC通过基于提示的因果引导在LDM分布内生成因果一致的视频反事实，在真实世界面部视频上实现了最先进的因果有效性，同时不损害时间一致性或视觉质量。由于其与任何黑盒视频编辑系统的兼容性，我们的框架在数字媒体和医疗保健等多个领域生成逼真的“如果”假设视频场景方面具有巨大潜力。
英文摘要：
Adapting text-to-image (T2I) latent diffusion models (LDMs) to video editing has shown strong visual fidelity and controllability, but challenges remain in maintaining causal relationships inherent to the video data generating process. Edits affecting causally dependent attributes often generate unrealistic or misleading outcomes if these relationships are ignored. In this work, we introduce a causally faithful framework for counterfactual video generation, formulated as an Out-of-Distribution (OOD) prediction problem. We embed prior causal knowledge by encoding the relationships specified in a causal graph into text prompts and guide the generation process by optimizing these prompts using a vision-language model (VLM)-based textual loss. This loss encourages the latent space of the LDMs to capture OOD variations in the form of counterfactuals, effectively steering generation toward causally meaningful alternatives. The proposed framework, dubbed CSVC, is agnostic to the underlying video editing system and does not require access to its internal mechanisms or fine-tuning. We evaluate our approach using standard video quality metrics and counterfactual-specific criteria, such as causal effectiveness and minimality. Experimental results show that CSVC generates causally faithful video counterfactuals within the LDM distribution via prompt-based causal steering, achieving state-of-the-art causal effectiveness without compromising temporal consistency or visual quality on real-world facial videos. Due to its compatibility with any black-box video editing system, our framework has significant potential to generate realistic 'what if' hypothetical video scenarios in diverse areas such as digital media and healthcare.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
