EgoFlow：基于梯度的流匹配用于第一人称6自由度物体运动生成
EgoFlow: Gradient-Guided Flow Matching for Egocentric 6DoF Object Motion Generation
AI总结：
本文提出EgoFlow，通过多模态第一人称观察生成物理合理的物体运动轨迹，结合Mamba-Transformer-Perceiver架构和梯度引导推理，提升运动生成的准确性和物理真实性。
摘要：
从第一人称视频理解和预测物体运动对于具身感知和交互至关重要。然而，生成物理一致的6自由度轨迹仍具挑战性，因遮挡、快速运动和现有生成模型缺乏显式物理推理。我们提出EgoFlow，一种流匹配框架，通过多模态第一人称观察合成真实且物理合理的轨迹。EgoFlow采用混合Mamba-Transformer-Perceiver架构，联合建模时间动态、场景几何和语义意图，同时梯度引导推理过程强制可微物理约束，如避障和运动平滑。这种组合产生连贯且可控的运动生成，无需后处理过滤或额外监督。在真实世界数据集HD-EPIC、EgoExo4D和HOT3D上的实验表明，EgoFlow在准确性、泛化能力和物理真实性上优于扩散和Transformer基线，碰撞率降低高达79%，并强泛化到未见场景。我们的结果展示了基于流的生成模型在可扩展且物理基础的第一人称运动理解中的潜力。
英文摘要：
Understanding and predicting object motion from egocentric video is fundamental to embodied perception and interaction. However, generating physically consistent 6DoF trajectories remains challenging due to occlusions, fast motion, and the lack of explicit physical reasoning in existing generative models. We present EgoFlow, a flow-matching framework that synthesizes realistic and physically plausible trajectories conditioned on multimodal egocentric observations. EgoFlow employs a hybrid Mamba-Transformer-Perceiver architecture to jointly model temporal dynamics, scene geometry, and semantic intent, while a gradient-guided inference process enforces differentiable physical constraints such as collision avoidance and motion smoothness. This combination yields coherent and controllable motion generation without post-hoc filtering or additional supervision. Experiments on real-world datasets HD-EPIC, EgoExo4D, and HOT3D show that EgoFlow outperforms diffusion-based and transformer baselines in accuracy, generalization, and physical realism, reducing collision rates by up to 79%, and strong generalization to unseen scenes. Our results highlight the promise of flow-based generative modeling for scalable and physically grounded egocentric motion understanding.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
