LAMP：语言辅助运动规划用于可控视频生成
LAMP: Language-Assisted Motion Planning for Controllable Video Generation
AI总结：
LAMP通过大语言模型生成3D轨迹，实现基于自然语言的视频生成，提升运动可控性和用户意图对齐。
摘要：
视频生成在视觉保真度和可控性方面取得显著进展，能够根据文本、布局或运动进行条件生成。其中，运动控制——指定物体动态和摄像机轨迹——对于构建复杂电影场景至关重要，但现有界面仍有限。我们介绍了LAMP，利用大语言模型（LLMs）作为运动规划器，将自然语言描述转换为动态物体和（相对定义）摄像机的显式3D轨迹。LAMP定义了一个运动领域特定语言（DSL），受电影摄影惯例启发。通过利用LLMs的程序合成能力，LAMP从自然语言生成结构化的运动程序，这些程序确定性地映射到3D轨迹。我们构建了一个大规模程序数据集，将自然文本描述与对应的运动程序和3D轨迹配对。实验表明，与现有最先进方法相比，LAMP在运动可控性和与用户意图对齐方面表现更优，建立了首个直接从自然语言规范生成物体和摄像机运动的框架。代码、模型和数据可在我们的项目页面上获得。
英文摘要：
Video generation has achieved remarkable progress in visual fidelity and controllability, enabling conditioning on text, layout, or motion. Among these, motion control - specifying object dynamics and camera trajectories - is essential for composing complex, cinematic scenes, yet existing interfaces remain limited. We introduce LAMP that leverages large language models (LLMs) as motion planners to translate natural language descriptions into explicit 3D trajectories for dynamic objects and (relatively defined) cameras. LAMP defines a motion domain-specific language (DSL), inspired by cinematography conventions. By harnessing program synthesis capabilities of LLMs, LAMP generates structured motion programs from natural language, which are deterministically mapped to 3D trajectories. We construct a large-scale procedural dataset pairing natural text descriptions with corresponding motion programs and 3D trajectories. Experiments demonstrate LAMP's improved performance in motion controllability and alignment with user intent compared to state-of-the-art alternatives establishing the first framework for generating both object and camera motions directly from natural language specifications. Code, models and data are available on our project page.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
