ReRoPE：将RoPE用于相对相机控制
ReRoPE: Repurposing RoPE for Relative Camera Control
AI总结：
ReRoPE通过将RoPE用于相对相机控制，实现可控视频生成，提升生成质量与效率。
摘要：
可控相机视角的视频生成对于交互内容创作、游戏和模拟等应用至关重要。现有方法通常通过将预训练视频模型适应到固定参考点的相机姿态相对位置上，例如第一帧。然而，这些编码缺乏位移不变性，常常导致泛化能力差和累积漂移。虽然在任意视角对之间定义的相对相机姿态嵌入提供了更稳健的替代方案，但将其整合到预训练视频扩散模型中而不带来显著的训练成本或架构变化仍具有挑战性。我们介绍了ReRoPE，这是一个即插即用的框架，它将相对相机信息整合到预训练视频扩散模型中，而不会影响其生成能力。我们的方法基于一个见解，即现有模型中的旋转位置嵌入（RoPE）在低频成分上未充分利用其完整的频谱带宽。通过无缝地将相对相机姿态信息注入这些未充分利用的频段中，ReRoPE实现了精确的控制，同时保持了强大的预训练生成先验。我们在图像到视频（I2V）和视频到视频（V2V）任务中评估了我们的方法，以评估相机控制精度和视觉保真度。我们的结果表明，ReRoPE提供了一条训练高效的方法，朝着可控、高保真视频生成的方向迈进。查看项目页面获取更多结果：https://sisyphe-lee.github.io/ReRoPE/
英文摘要：
Video generation with controllable camera viewpoints is essential for applications such as interactive content creation, gaming, and simulation. Existing methods typically adapt pre-trained video models using camera poses relative to a fixed reference, e.g., the first frame. However, these encodings lack shift-invariance, often leading to poor generalization and accumulated drift. While relative camera pose embeddings defined between arbitrary view pairs offer a more robust alternative, integrating them into pre-trained video diffusion models without prohibitive training costs or architectural changes remains challenging. We introduce ReRoPE, a plug-and-play framework that incorporates relative camera information into pre-trained video diffusion models without compromising their generation capability. Our approach is based on the insight that Rotary Positional Embeddings (RoPE) in existing models underutilize their full spectral bandwidth, particularly in the low-frequency components. By seamlessly injecting relative camera pose information into these underutilized bands, ReRoPE achieves precise control while preserving strong pre-trained generative priors. We evaluate our method on both image-to-video (I2V) and video-to-video (V2V) tasks in terms of camera control accuracy and visual fidelity. Our results demonstrate that ReRoPE offers a training-efficient path toward controllable, high-fidelity video generation. See project page for more results: https://sisyphe-lee.github.io/ReRoPE/
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
