Frame In-N-Out：无界可控的图像到视频生成
Frame In-N-Out: Unbounded Controllable Image-to-Video Generation
AI总结：
针对视频生成中的可控性、时间一致性和细节合成难题，本文提出基于帧入帧出技术的新数据集、身份保持的运动可控视频扩散Transformer架构及评估协议，显著优于现有基线。
摘要：
可控性、时间一致性和细节合成仍然是视频生成中最关键的挑战。本文聚焦于一种常用但未被充分探索的电影技术，即“帧入帧出”（Frame In and Frame Out）。具体而言，从图像到视频生成出发，用户可以通过指定的运动轨迹，控制图像中的物体自然离开场景，或提供全新的身份参考以进入场景。为支持该任务，我们引入了一个半自动构建的新数据集、一种高效的保持身份且运动可控的视频扩散Transformer架构，以及一套针对该任务的全面评估协议。我们的评估表明，所提方法显著优于现有基线。
英文摘要：
Controllability, temporal coherence, and detail synthesis remain the most critical challenges in video generation. In this paper, we focus on a commonly used yet underexplored cinematic technique known as Frame In and Frame Out. Specifically, starting from image-to-video generation, users can control the objects in the image to naturally leave the scene or provide breaking new identity references to enter the scene, guided by a user-specified motion trajectory. To support this task, we introduce a new dataset that is curated semi-automatically, an efficient identity-preserving motion-controllable video Diffusion Transformer architecture, and a comprehensive evaluation protocol targeting this task. Our evaluation shows that our proposed approach significantly outperforms existing baselines.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
