联合流轨迹优化：从视频演示生成可行的机器人运动
Joint Flow Trajectory Optimization For Feasible Robot Motion Generation from Video Demonstrations
AI总结：
提出联合流轨迹优化（JFTO）框架，通过以物体为中心的引导和SE(3)流匹配，在视频演示下实现可行的抓取姿态生成与物体轨迹模仿，兼顾抓取相似性、轨迹似然和碰撞惩罚。
摘要：
从人类视频演示中学习为遥操作或动觉教学提供了一种可扩展的替代方案，但由于具身差异和关节可行性约束，给机器人操作器带来了挑战。我们通过提出联合流轨迹优化（JFTO）框架来解决这一问题，该框架用于基于视频的从演示学习（LfD）范式下的抓取姿态生成和物体轨迹模仿。我们的方法不是直接模仿人类手部运动，而是将演示视为以物体为中心的引导，平衡三个目标：（i）选择可行的抓取姿态，（ii）生成与演示运动一致的物体轨迹，以及（iii）确保在机器人运动学内的无碰撞执行。为了捕捉演示的多模态特性，我们将流匹配扩展到SE(3)以对物体轨迹进行概率建模，从而实现避免模式坍缩的密度感知模仿。由此产生的优化将抓取相似性、轨迹似然和碰撞惩罚整合到一个统一的可微目标中。我们在仿真和真实世界实验中验证了我们的方法，涵盖了多种真实世界的操作任务。
英文摘要：
Learning from human video demonstrations offers a scalable alternative to teleoperation or kinesthetic teaching, but poses challenges for robot manipulators due to embodiment differences and joint feasibility constraints. We address this problem by proposing the Joint Flow Trajectory Optimization (JFTO) framework for grasp pose generation and object trajectory imitation under the video-based Learning-from-Demonstration (LfD) paradigm. Rather than directly imitating human hand motions, our method treats demonstrations as object-centric guides, balancing three objectives: (i) selecting a feasible grasp pose, (ii) generating object trajectories consistent with demonstrated motions, and (iii) ensuring collision-free execution within robot kinematics. To capture the multimodal nature of demonstrations, we extend flow matching to $\SE(3)$ for probabilistic modeling of object trajectories, enabling density-aware imitation that avoids mode collapse. The resulting optimization integrates grasp similarity, trajectory likelihood, and collision penalties into a unified differentiable objective. We validate our approach in both simulation and real-world experiments across diverse real-world manipulation tasks.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
