MotionBooth：具备运动感知能力的定制化文本生成视频方法
MotionBooth: Motion-Aware Customized Text-to-Video Generation
AI总结：
本研究提出MotionBooth定制化文本生成视频框架，通过少量图像微调模型，结合多种损失函数与免训练运动控制技术，实现主体与相机运动的精准控制，同时保留主体外观，经多维度评估验证了方法的有效性。
摘要：
本研究提出了MotionBooth这一创新框架，旨在实现定制化主体的动态生成，同时对物体运动与相机运动实现精准控制。仅需特定物体的少量图像，我们就能高效微调文本生成视频模型，精准捕捉物体的外形与属性。该方法设计了主体区域损失与视频保留损失以提升主体的学习效果，还引入主体token交叉注意力损失，将定制化主体与运动控制信号相融合。此外，我们提出了推理阶段无需训练的技术，用于管控主体运动与相机运动：具体而言，利用交叉注意力图操控来调控主体运动，同时提出一种新型的潜在位移模块（latent shift module）用于控制相机运动。MotionBooth在控制生成视频中运动的同时，能出色保留主体的外观。大量定量与定性评估均验证了该方法的优越性与有效性。项目页面位于：https://jianzongwu.github.io/projects/motionbooth
英文摘要：
In this work, we present MotionBooth, an innovative framework designed for animating customized subjects with precise control over both object and camera movements. By leveraging a few images of a specific object, we efficiently fine-tune a text-to-video model to capture the object's shape and attributes accurately. Our approach presents subject region loss and video preservation loss to enhance the subject's learning performance, along with a subject token cross-attention loss to integrate the customized subject with motion control signals. Additionally, we propose training-free techniques for managing subject and camera motions during inference. In particular, we utilize cross-attention map manipulation to govern subject motion and introduce a novel latent shift module for camera movement control as well. MotionBooth excels in preserving the appearance of subjects while simultaneously controlling the motions in generated videos. Extensive quantitative and qualitative evaluations demonstrate the superiority and effectiveness of our method. Our project page is at https://jianzongwu.github.io/projects/motionbooth
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
