arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2610.14693cs.AI
收藏

AcrossVAM2.0:面向参数高效视频扩散适配的冲突感知点监督

AcrossVAM2.0: Conflict-Aware Point Supervision for Parameter-Efficient Video Diffusion Adaptation

Across Physcis (Beijing) Technology Co., Ltd(跨物理(北京)科技有限公司)

Nan Wu, Yee Hung Hong

AI总结:

该研究针对50.1亿参数的文本到视频扩散Transformer,提出AcrossVAM2.0冲突感知点监督方法,结合置信度加权跟踪等技术,在降低KLT等循环误差的同时未造成广泛感知退化,相关成果已公开。

摘要:

显式点对应关系对视频生成器的适配颇具吸引力:它描述了可见表面的运动方式,但可能会覆盖大型预训练模型学习到的外观先验。我们在一个拥有50.1亿参数的文本到视频扩散Transformer中研究了这种张力。ACROSSVAM2.0结合了置信度加权的跟踪与可见性头、零初始化残差反馈,以及非对称PCGrad,该算法仅移除共享LoRA参数上与主流匹配梯度相对立的辅助组件。该设计在初始化时精确保留生成器,并在适配过程中保护其主更新。在一项训练1000步、包含3个独立随机种子的四向消融实验中,与匹配的仅流延续方法相比,完整方法在所有3个种子中均降低了KLT前向-后向循环误差(种子级平均差值为-0.00711,标准差为0.00505),同时鲁棒清晰度在所有3个种子中均保持为正。一项扩展的24提示评估(216个视频)显示,与匹配对照组相比,其聚合CoTracker3循环误差降低了9.7%,且在6个VBench维度上的性能接近匹配;不过,种子级跟踪效果存在异质性,I3D-FVD从0.1824退化为0.2022。这些结果支持一个有限结论:冲突感知适配可提升对应一致性,且不会造成广泛的感知退化,但无法统一提升视频质量。我们发布了相关公式、伪代码、匹配输出、源表格及工件来源。

英文摘要:

Explicit point correspondence is attractive for adapting video generators: it describes how visible surfaces move, but can overwrite the appearance prior learned by a large pretrained model. We study this tension in a 5.01B-parameter text-to-video diffusion transformer. ACROSSVAM2.0 combines a confidence-weighted track-and-visibility head, zero-initialized residual feedback, and asymmetric PCGrad that removes only the auxiliary component opposing the primary flow-matching gradient on shared LoRA parameters. The design preserves the generator exactly at initialization and protects its primary update during adaptation. In a four-way ablation trained for 1,000 steps with three independent seeds, the full method lowers KLT forward-backward cycle error in all three seeds relative to a matched flow-only continuation (mean seed-level difference -0.00711, SD 0.00505), while robust sharpness remains positive in all three. An expanded 24-prompt evaluation (216 videos) yields 9.7% lower aggregate CoTracker3 cycle error than the matched control and near-matched performance on six VBench dimensions; however, the seed-level tracking effect is heterogeneous and I3D-FVD regresses from 0.1824 to 0.2022. These results support a bounded claim: conflict-aware adaptation improves correspondence consistency without broad perceptual degradation, but does not uniformly improve video quality. We release equations, pseudocode, matched outputs, source tables, and artifact provenance.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑