DiReCT:对比轨迹解耦的对比正则化用于物理精细视频生成
DiReCT: Disentangled Regularization of Contrastive Trajectories for Physics-Refined Video Generation
AI总结:
本文提出DiReCT方法,通过解耦对比轨迹正则化提升物理精细视频生成的物理合理性,改进VideoPhy的物理常识评分。
摘要:
流匹配视频生成器能够产生时间上连贯、高保真的输出,但经常违反基本物理定律,因为其重建目标惩罚单帧偏差而不区分物理一致的动力学与不可能的动力学。对比流匹配提供了一种系统性的解决方法,通过推远不同条件下的速度场轨迹,但我们发现文本条件视频设置中存在一个根本性的障碍:语义-物理纠缠。由于自然语言提示将场景内容与物理行为耦合,朴素的负采样会抽取出速度场与正样本 largely 重叠的条件,导致对比梯度直接反对流匹配目标。我们正式化了这种梯度冲突,推导出一个精确的对齐条件,揭示了对比学习在何时有助于训练,何时会损害训练。基于此分析,我们引入DiReCT(对比轨迹解耦的对比正则化),一个轻量级的后训练框架,将对比信号分解为两个互补的尺度:一个宏观对比项,从语义上遥远的区域抽取分区排斥的负样本以实现干扰-free 的全局轨迹分离,以及一个微观对比项,构建与正样本共享完整场景语义但沿单一LLM扰动轴的物理行为不同的硬负样本;涵盖运动学、力、材料、相互作用和量级。一个速度空间分布正则化器有助于防止预训练视觉质量的灾难性遗忘。当应用于Wan 2.1-1.3B时,我们的方法在VideoPhy上将物理常识评分提高16.7%和11.3%,分别优于基线和SFT,而无需增加训练时间。
英文摘要:
Flow-matching video generators produce temporally coherent, high-fidelity outputs yet routinely violate elementary physics because their reconstruction objectives penalize per-frame deviations without distinguishing physically consistent dynamics from impossible ones. Contrastive flow matching offers a principled remedy by pushing apart velocity-field trajectories of differing conditions, but we identify a fundamental obstacle in the text-conditioned video setting: semantic-physics entanglement. Because natural-language prompts couple scene content with physical behavior, naive negative sampling draws conditions whose velocity fields largely overlap with the positive sample's, causing the contrastive gradient to directly oppose the flow-matching objective. We formalize this gradient conflict, deriving a precise alignment condition that reveals when contrastive learning helps versus harms training. Guided by this analysis, we introduce DiReCT (Disentangled Regularization of Contrastive Trajectories), a lightweight post-training framework that decomposes the contrastive signal into two complementary scales: a macro-contrastive term that draws partition-exclusive negatives from semantically distant regions for interference-free global trajectory separation, and a micro-contrastive term that constructs hard negatives sharing full scene semantics with the positive sample but differing along a single, LLM-perturbed axis of physical behavior; spanning kinematics, forces, materials, interactions, and magnitudes. A velocity-space distributional regularizer helps to prevent catastrophic forgetting of pretrained visual quality. When applied to Wan 2.1-1.3B, our method improves the physical commonsense score on VideoPhy by 16.7% and 11.3% compared to the baseline and SFT, respectively, without increasing training time.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。