PoseTraj：视频扩散中的姿态感知轨迹控制
PoseTraj: Pose-Aware Trajectory Control in Video Diffusion
AI总结：
针对现有轨迹引导视频生成模型3D理解不足、难生成大角度旋转下6D姿态变化物体运动的问题，提出PoseTraj模型与两阶段预训练框架，通过合成数据集与3D边界框监督等提升性能，在姿态对齐与质量上优于基线。
摘要：
轨迹引导的视频生成领域近期已取得显著进展，但受限于3D理解能力不足，现有模型在生成大角度旋转下6D姿态可能变化的物体运动时仍面临挑战。为解决该问题，我们提出PoseTraj——一种姿态感知的视频拖拽模型，用于从2D轨迹生成符合3D对齐的运动。\n我们的方法采用新颖的两阶段姿态感知预训练框架，提升模型在多样轨迹下的3D理解能力。具体而言，我们构建了大规模合成数据集PoseTraj-10K，包含1万个物体沿旋转轨迹运动的视频，并引入3D bounding box（边界框）作为中间监督信号，增强模型对物体姿态变化的感知。在此基础上，我们在真实视频上微调轨迹控制模块，并加入额外的相机解耦模块以进一步提升运动精度。\n在多个基准数据集上的实验表明，我们的方法不仅在旋转轨迹的3D姿态对齐拖拽任务中表现优异，还在轨迹精度和视频质量方面优于现有基线模型。
英文摘要：
Recent advancements in trajectory-guided video generation have achieved notable progress. However, existing models still face challenges in generating object motions with potentially changing 6D poses under wide-range rotations, due to limited 3D understanding. To address this problem, we introduce PoseTraj, a pose-aware video dragging model for generating 3D-aligned motion from 2D trajectories. Our method adopts a novel two-stage pose-aware pretraining framework, improving 3D understanding across diverse trajectories. Specifically, we propose a large-scale synthetic dataset PoseTraj-10K, containing 10k videos of objects following rotational trajectories, and enhance the model perception of object pose changes by incorporating 3D bounding boxes as intermediate supervision signals. Following this, we fine-tune the trajectory-controlling module on real-world videos, applying an additional camera-disentanglement module to further refine motion accuracy. Experiments on various benchmark datasets demonstrate that our method not only excels in 3D pose-aligned dragging for rotational trajectories but also outperforms existing baselines in trajectory accuracy and video quality.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
