arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2606.22835cs.CVcs.AI
收藏

OrthoMotion:通过几何语义正交注意力解耦相机与主体运动

OrthoMotion:Disentangling Camera and Subject Motion via Geometry Semantics Orthogonal Attention

Peking University(北京大学)

Zijie Meng

AI总结:

提出OrthoMotion方法,通过将相机运动编码为几何通道(旋转位置嵌入相位旋转)和主体运动编码为语义通道(门控值注入交叉注意力),并利用正交正则化保证解耦,实现可控视频生成中相机与主体运动的独立控制,将串扰降低2.4倍以上。

摘要:

可控视频生成要求独立控制相机和主体,但二维条件将它们纠缠在一起:相机和物体引起的光流共享相同的逆深度(1/Z)缩放,无法仅从图像证据中分离。我们首先证明这种纠缠是表征性的而非架构性的——二维相机/物体分割是一个不可识别的逆问题——因此将解耦重新定义为算子设计问题。我们在注意力算子层面解决它。OrthoMotion将相机运动路由到几何通道(旋转位置嵌入(RoPE)相位的保范旋转)和主体运动路由到语义通道(交叉注意力中的门控值注入)。由于这些子算子在代数上是互补的——对令牌的仿射作用的旋转与平移——一个轻量级的解耦正则化器可证明地将它们的响应子空间驱动到正交,从而两种控制停止相互干扰。据我们所知,OrthoMotion是第一种通过构造而非期望来保证解耦的方法。它同时达到了最先进的相机和主体精度,同时最小化串扰,我们使用新的串扰误差(CTE)指标量化了这一点,将串扰降低了2.4倍以上,且保真度无损失,并跨骨干网络泛化。

英文摘要:

Controllable video generation demands independent command of the camera and the subject, yet 2D conditioning entangles them: camera- and object-induced optical flow share the same inverse-depth (1/Z) scaling and cannot be separated from image evidence alone. We first prove that this entanglement is representational, not architectural -- the 2D camera/object split is a non-identifiable inverse problem -- and therefore reframe decoupling as a question of operator design. We resolve it at the level of the attention operator. OrthoMotion routes camera motion into a geometric channel, a norm-preserving rotation of the rotary position embedding (RoPE) phase, and subject motion into a semantic channel, a gated value injection in cross-attention. Because these sub-operators are algebraically complementary -- a rotation versus a translation of the affine action on tokens -- a lightweight decoupling regularizer provably drives their response subspaces to orthogonality, so the two controls stop interfering. To our knowledge OrthoMotion is the first method to guarantee disentanglement by construction rather than hope for it to emerge. It attains state-of-the-art camera and subject accuracy at once while minimizing cross-talk, which we quantify with a new Cross-Talk Error (CTE) metric, cutting cross-talk by more than 2.4x with no loss in fidelity and generalizing across backbones.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑