从同步到序列：通过插值实现外视角到视角的视频生成
From Synchrony to Sequence: Exo-to-Ego Generation via Interpolation
AI总结：
本文提出Syn2Seq-Forcing方法，通过插值生成连续信号解决外视角到视角视频生成中的时空和几何断点问题，提升扩散模型在跨视角视频合成中的性能。
摘要：
Exo-to-Ego视频生成旨在从同步的第三方视角和对应的摄像机姿态合成第一人称视频。尽管有配对监督，同步的外-视角数据本质上引入了显著的时空和几何不连续性，违反了标准视频生成基准的平滑运动假设。我们识别出这种同步引起的跳跃是核心挑战，并提出Syn2Seq-Forcing，一种通过插值源和目标视频形成单一连续信号的序列化方法。通过将Exo2Ego重新表述为序列信号建模而非传统条件-输出任务，我们的方法使基于扩散的序列模型（如扩散强制变换器DFoT）能够更有效地捕捉跨帧的连贯过渡。实验表明，仅插值视频而不进行姿态插值已产生显著改进，强调了时空不连续性是主导困难。除了即时性能提升外，这种方法建立了一个通用且灵活的框架，能够统一外-视角和视角-外生成于单一连续序列模型中，为跨视角视频合成的未来研究提供了原理性基础。
英文摘要：
Exo-to-Ego video generation aims to synthesize a first-person video from a synchronized third-person view and corresponding camera poses. While paired supervision is available, synchronized exo-ego data inherently introduces substantial spatio-temporal and geometric discontinuities, violating the smooth-motion assumptions of standard video generation benchmarks. We identify this synchronization-induced jump as the central challenge and propose Syn2Seq-Forcing, a sequential formulation that interpolates between the source and target videos to form a single continuous signal. By reframing Exo2Ego as sequential signal modeling rather than a conventional condition-output task, our approach enables diffusion-based sequence models, e.g. Diffusion Forcing Transformers (DFoT), to capture coherent transitions across frames more effectively. Empirically, we show that interpolating only the videos, without performing pose interpolation already produces significant improvements, emphasizing that the dominant difficulty arises from spatio-temporal discontinuities. Beyond immediate performance gains, this formulation establishes a general and flexible framework capable of unifying both Exo2Ego and Ego2Exo generation within a single continuous sequence model, providing a principled foundation for future research in cross-view video synthesis.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
