arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2610.41540cs.LGcs.RO
收藏

FOCUS:基于最优传输监督的一阶控制用于观测模仿

FOCUS: First-Order Control Using Optimal Transport Supervision for Imitation from Observations

University of Geneva(日内瓦大学);University of Applied Sciences and Arts Western Switzerland(瑞士西部应用科学与艺术大学)

João A. Cândido Ramos, Lionel Blondé, Hugues Vinzant, Jacopo Castellini, Alexandros Kalousis

AI总结:

提出FOCUS算法,结合局部最优传输对齐与短视域可微演员-评论家学习,在DFlex的含接触运动任务中,相比同类方法提升了样本效率与训练稳定性。

摘要:

从观测中进行模仿学习需仅通过状态演示恢复行为,且无已知时间对应关系。可微模拟器提供路径策略梯度,但对长且含接触的轨迹求导会使优化不稳定。我们提出FOCUS,其结合局部最优传输对齐与短视域可微演员-评论家学习。FOCUS采用软K选最优目标,将智能体展开窗口与多个采样的专家片段匹配,该目标强调局部兼容匹配。所得基于传输的代价提供可微模仿奖励,而学习到的评论家则在展开视域外引导价值。FOCUS无需时间对齐轨迹、专家对齐初始化或教师强制。在DFlex中含接触的运动任务(包括由152个肌肉激活控制的肌肉骨骼人形机器人)上,FOCUS相比对抗式观测模仿方法及先前可微模仿基线,提升了样本效率与训练稳定性。

英文摘要:

Imitation learning from observations must recover behavior from state-only demonstrations without known temporal correspondence. Differentiable simulators provide pathwise policy gradients, but differentiating through long, contact-rich trajectories can make optimization unstable. We introduce FOCUS, which combines local optimal-transport alignment with short-horizon differentiable actor-critic learning. FOCUS matches agent rollout windows to multiple sampled expert segments using a soft best-of-$K$ objective that emphasizes locally compatible matches. The resulting transport-based costs provide differentiable imitation rewards, while a learned critic bootstraps value beyond the rollout horizon. FOCUS requires neither temporally aligned trajectories nor expert-aligned initialization or teacher forcing. Across contact-rich locomotion tasks in DFlex, including a musculoskeletal humanoid controlled by 152 muscle activations, FOCUS improves sample efficiency and training stability over adversarial imitation-from-observation methods and prior differentiable imitation baselines.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑