MotionCoach：为视频专业动作编辑构建多模态引导
MotionCoach: Constructing Multimodal Guidance for Video Professional Motion Editing
AI总结：
MotionCoach是一种多模态智能体，通过构建多模态引导解决专业动作编辑难题，在Seedance 2.0和MiniMax-H3上分别提升了视频编辑模型的总体得分。
摘要：
视频编辑模型凭借大规模训练数据，在修改外观、背景、风格和动作方面表现出色。除通用场景外，专业动作在表演、体育和武术领域意义重大。本文聚焦专业动作编辑，其涉及精细的动作技巧与时间结构。给定专业动作名称但缺乏详细描述时，现有模型常无法生成专业动作，例如在“翘前轮特技（Stoppie）”指令下错误抬起前轮。为应对这一挑战，我们提出MotionCoach，这是一个经过训练的多模态智能体，使用多轮工具并执行自我验证，生成基于事实的提示和参考片段，作为可靠且充足的多模态引导，将外部专业动作知识注入视频编辑模型。为提供训练数据，我们构建了数据管道以生成轨迹。我们在MotionCoach-SFT-10k上冷启动该智能体，并在MotionCoach-RL-2k上应用分组相对策略优化（GRPO），奖励来自动作引导提示、动作结果视频和格式。为进行评估，我们引入VPMEBench，包含314项带有精细参考和注释的专业动作编辑任务。大量实验表明，MotionCoach在Seedance 2.0上将总体得分从55.1提升至65.6，在MiniMax-H3上从51.2提升至64.5。学习构建多模态引导可实现精确的专业动作编辑，无需重新训练视频编辑模型。
英文摘要：
Video editing models achieve strong performance in modifying appearance, backgrounds, styles, and motions, empowered by large-scale training data. Beyond general scenarios, professional motions are significant in performance, sports, and martial arts. This paper focuses on professional motion editing, which involves fine-grained motion technique and temporal structure. Given professional motion names but lacking detailed descriptions, existing models often fail to generate professional motions, such as incorrectly lifting the front wheel under a Stoppie instruction. To address this challenge, we introduce MotionCoach, a trained multimodal agent that uses multi-turn tools and performs self-verification, producing a grounded prompt and a reference clip as reliable and sufficient multimodal guidance to inject external professional motion knowledge into video editing models. To supply training data, we build a data pipeline to construct trajectories. We cold-start the agent on MotionCoach-SFT-10k and apply Group Relative Policy Optimization (GRPO) on MotionCoach-RL-2k with rewards on the motion guidance prompt, the motion outcome video, and the format. For evaluation, we introduce VPMEBench, comprising 314 professional motion editing tasks with fine-grained references and annotations. Extensive experiments show that MotionCoach improves the overall score from 55.1 to 65.6 on Seedance 2.0 and from 51.2 to 64.5 on MiniMax-H3. Learning to construct multimodal guidance enables precise professional motion editing without retraining video editing models.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
