发表机构
California Institute of Technology（加州理工学院）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
DynaTokens通过轻量级可学习标记，在测试时为相机控制视频模型注入场景动态，实现动态与相机运动的更好权衡。
AI 中文摘要
视频生成必须考虑两种运动来源：一种由观察者的相机路径引起，另一种由场景动态引起。理想的相机控制视频模型应同时处理这两种运动：让用户在移动相机的同时演化场景动态。虽然当前模型在静态设置中能很好地处理相机引起的运动，但在动态场景中却表现不佳：物体静止不动、运动不正确或生成质量下降。我们引入了DynaTokens，一组轻量级的可学习场景特定标记，向现有的相机控制世界模型教授动态。我们的方法基于两种运动来源之间的简单不对称性：相机运动全局影响生成的视图，而物体动态在空间上是局部的。通过交叉注意力，DynaTokens在保持基础模型冻结的情况下，从场景的少量示例轨迹中训练可学习标记，并在新的查询相机路径下实现动态。DynaTokens在VBench2和WorldScore评估中实现了比LoRA、块微调和专门的训练层基线更好的动态-相机同时权衡。对标记注意力、消融和运动时间性的分析表明，将可训练接口与学习目标的结构相匹配对于有效适应很重要。项目网站：此https URL
英文摘要
Video generation must account for two sources of motion, one induced by the observer's camera path and the other caused by scene dynamics. An ideal camera-controlled video model should account for both motions: let users move the camera while evolving the scene dynamics. While current models handle camera-induced motion well in static settings, they struggle for dynamic scenes: objects are static, move incorrectly, or degrade in generation quality. We introduce DynaTokens, a lightweight set of learnable scene-specific tokens that teach dynamics to an existing camera-controlled world model. Our method is motivated by a simple asymmetry between the two sources of motion: whereas camera motion affects the generated view globally, object dynamics are spatially localized. Through cross-attention, DynaTokens trains the learnable tokens from a few example trajectories for a scene while keeping the base model frozen, and enables dynamics under new query camera paths. DynaTokens achieves a better simultaneous dynamics-camera tradeoff on VBench2 and WorldScore evaluations than LoRA, block finetuning, and specialized trainable-layer baselines. Analyses of token attention, ablations, and motion temporality suggest that matching the trainable interface to the structure of the learning target is important for effective adaptation. Project website: https://glab-caltech.github.io/dynatokens/
CommentsProject website: https://glab-caltech.github.io/dynatokens/
