在这项工作中，我们提出了一种用于视频生成中运动定制化的新方法，填补了视频生成模型在运动表示探索方面普遍存在的空白。考虑到视频的时空特性所带来的独特挑战，我们的方法引入了运动嵌入（Motion Embeddings），这是一组从给定视频中推导出的显式、时间一致的嵌入。这些嵌入旨在与视频扩散模型的时间 Transformer 模块无缝集成，在不损害空间完整性的前提下，跨帧调制自注意力计算。我们的方法为运动表示提供了一种紧凑且高效的解决方案，利用两种类型的嵌入：运动查询-键嵌入（Motion Query-Key Embedding）用于调制时间注意力图，运动值嵌入（Motion Value Embedding）用于调制注意力值。此外，我们引入了一种推理策略，该策略从运动查询-键嵌入中排除空间维度，并对运动值嵌入应用差分运算，两者都旨在去除外观偏置，确保嵌入仅关注运动。我们的贡献包括引入一种专为定制化任务设计的运动嵌入，并通过大量实验证明了我们方法的实际优势和有效性。
英文摘要：
In this work, we present a novel approach for motion customization in video generation, addressing the widespread gap in the exploration of motion representation within video generative models. Recognizing the unique challenges posed by the spatiotemporal nature of video, our method introduces Motion Embeddings, a set of explicit, temporally coherent embeddings derived from a given video. These embeddings are designed to integrate seamlessly with the temporal transformer modules of video diffusion models, modulating self-attention computations across frames without compromising spatial integrity. Our approach provides a compact and efficient solution to motion representation, utilizing two types of embeddings: a Motion Query-Key Embedding to modulate the temporal attention map and a Motion Value Embedding to modulate the attention values. Additionally, we introduce an inference strategy that excludes spatial dimensions from the Motion Query-Key Embedding and applies a differential operation to the Motion Value Embedding, both designed to debias appearance and ensure the embeddings focus solely on motion. Our contributions include the introduction of a tailored motion embedding for customization tasks and a demonstration of the practical advantages and effectiveness of our method through extensive experiments.
MotionAdapter: Video Motion Transfer via Content-Aware Attention Customization
Hangzhou Dianzi University（杭州电子科技大学）；Harbin Institute of Technology (Shenzhen)（哈尔滨工业大学（深圳））；The Hong Kong University of Science and Technology（香港科技大学）；Ocean University of China（中国海洋大学）；Singapore Management University（新加坡管理大学）
OmniVCus: Feedforward Subject-driven Video Customization with Multimodal Control Conditions
Johns Hopkins University（约翰霍普金斯大学）；Adobe Research（Adobe研究）；The University of Hong Kong（香港大学）；The Chinese University of Hong Kong（香港中文大学）；Shanghai Jiao Tong University（上海交通大学）
SketchAnimator: Animate Sketch via Motion Customization of Text-to-Video Diffusion Models
PRIS, School of Artificial Intelligence, Beijing University of Posts（PRIS人工智能学院，北京邮电大学）；SketchX, CVSSP, University of Surrey（SketchX，计算机视觉与模式识别研究所，萨里大学）
VMC: Video Motion Customization using Temporal Attention Adaption for Text-to-Video Diffusion Models
Kim Jaechul Graduate School of AI（金宰澈人工智能研究生院）；Bio and Brain Engineering（生物与脑工程系）；Korea Advanced Institute of Science and Technology (KAIST)（韩国科学技术院（KAIST））
Delving into the Temporal Challenges of Unified Video Protection Against Image-to-Video and Fine-Tuning-based Customization
The University of Sydney（悉尼大学）；University of Melbourne（墨尔本大学）；City University of Hong Kong（香港城市大学）；Wuhan University（武汉大学）；Mohamed bin Zayed University of Artificial Intelligence（穆罕默德·本·扎耶德人工智能大学）
DreamVideo-Omni: Omni-Motion Controlled Multi-Subject Video Customization with Latent Identity Reinforcement Learning
Fudan University（复旦大学）；The Hong Kong University of Science and Technology（香港科技大学）；Tongyi Lab, Alibaba Group（阿里云实验室）；Zhejiang University（浙江大学）；MMLab, The University of Hong Kong（香港大学MMLab）；Nanyang Technological University（南洋理工大学）；Show Lab, National University of Singapore（新加坡国立大学Show Lab）
NewMove: Customizing text-to-video models with novel motions
Massachusetts Institute of Technology（麻省理工学院）；Adobe Research（奥多比研究中心）；Czech Institute of Informatics, Robotics and Cybernetics at the Czech Technical University in Prague（布拉格捷克技术大学捷克信息学、机器人与控制研究所）
CusEnhancer: A Zero-Shot Scene and Controllability Enhancement Method for Photo Customization via ResInversion
College of Computer Science and Technology, East China University of Science and Technology（计算机科学与技术学院，东华大学）；Carnegie Mellon University（卡内基梅隆大学）；Microsoft（微软公司）
When Distillation Breaks Motion Control: Restoring Generative Trajectories for Fast Video Generators
Zhejiang University of Technology（浙江工业大学）；University of New South Wales (UNSW Sydney)（新南威尔士大学（悉尼））；University of Auckland（奥克兰大学）；Zhejiang University（浙江大学）
MotionCrafter: One-Shot Motion Customization of Diffusion Models
MAIS, Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所多模态人工智能系统全国重点实验室）；School of AI, UCAS（中国科学院大学人工智能学院）；Institute of Computing Technology, Chinese Academy of Sciences（中国科学院计算技术研究所）；Kuaishou Technology（快手科技）
Edit-VAR: Taming Visual Autoregressive Model for Precise Video Editing
Sun Yat-sen University（中山大学）；South China University of Technology（华南理工大学）；Tsinghua University（清华大学）；Shandong University（山东大学）；Nankai University（南开大学）；Hunan University（湖南大学）
JointTuner: Appearance-Motion Adaptive Joint Training for Customized Video Generation
College of Computer Science and Technology, National University of Defense Technology（计算机科学与技术学院，国防科技大学）；Alibaba International Digital Commerce（阿里巴巴国际数字商业）
VideoDirector: Precise Video Editing via Text-to-Video Models
Shenzhen Campus of Sun Yat-sen University（中山大学深圳校区）；Sun Yat-sen University（中山大学）；Tsinghua University（清华大学）；National University of Defense Technology（国防科技大学）
Tuning-Free Visual Customization via View Iterative Self-Attention Control
Tsinghua University（清华大学）；Tsinghua Shenzhen International Graduate School（清华大学深圳国际研究生院）；Nanchang University（南昌大学）；Jiluan Academy, Nanchang University（南昌大学吉鸾学院）；The University of Texas at Austin（德克萨斯大学奥斯汀分校）
针对 LoRA 等方法依赖训练和多张参考图的问题，本文提出免训练的 VisCtrl，通过 DDIM 反演与迭代自注意力特征注入，实现单参考图的图像、视频及 3D 场景个性化编辑。
