arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2602.13185cs.CVcs.GR

FlexAM: 一种灵活的外观-运动分解方法用于多功能视频生成控制

FlexAM: Flexible Appearance-Motion Decomposition for Versatile Video Generation Control

HKUST(GZ)(香港科技大学(广州));HKUST(香港科技大学);MUST(澳门大学);Tsinghua University(清华大学)

Mingzhi Sheng, Zekai Gu, Peng Li, Cheng Lin, Hao-Xiang Guo, Ying-Cong Chen, Yuan Liu

AI总结:

FlexAM通过引入新型3D控制信号,实现外观与运动的解耦,提升视频生成任务的灵活性和生成质量。

摘要:

在视频生成中实现有效且可推广的控制仍然是一个重大挑战。尽管许多方法依赖于模糊或任务特定的信号,我们认为对“外观”和“运动”的根本解耦提供了更稳健和可扩展的路径。我们提出了FlexAM,一种基于新型3D控制信号的统一框架。该信号将视频动态表示为点云,引入了三个关键增强:多频位置编码以区分细粒度运动、深度感知的位置编码,以及一个灵活的控制信号以在精度和生成质量之间取得平衡。这种表示方法使FlexAM能够有效解耦外观和运动,使包括I2V/V2V编辑、摄像机控制和空间物体编辑在内的广泛任务成为可能。大量实验表明,FlexAM在所有评估任务中均实现了优越的性能。

英文摘要:

Effective and generalizable control in video generation remains a significant challenge. While many methods rely on ambiguous or task-specific signals, we argue that a fundamental disentanglement of "appearance" and "motion" provides a more robust and scalable pathway. We propose FlexAM, a unified framework built upon a novel 3D control signal. This signal represents video dynamics as a point cloud, introducing three key enhancements: multi-frequency positional encoding to distinguish fine-grained motion, depth-aware encoding, and a flexible control signal for balancing precision and generalization. This representation allows FlexAM to effectively disentangle appearance and motion, enabling a wide range of tasks including I2V/V2V editing, camera control, and spatial object editing. Extensive experiments demonstrate that FlexAM achieves superior performance across all evaluated tasks.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑