arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2605.30503cs.ROcs.SYeess.SYstat.ML
收藏

混合接触动力学下的物理信息目标条件强化学习

Physics-informed Goal-Conditioned Reinforcement Learning under Hybrid Contact Dynamics

Department of Computer Science(计算机科学系)

Vittorio Giammarino, Anastasios Manganaris, Ahmed H. Qureshi

AI总结:

针对接触丰富任务中混合动力学导致现有物理信息目标条件强化学习方法性能下降的问题,提出接触感知和分层公式,选择性应用物理信息归纳偏置,向接触丰富操作扩展。

摘要:

从稀疏反馈中学习达到任意目标需要智能体推断状态-目标对之间的丰富可达性概念。目标条件强化学习(GCRL)通过学习跨目标泛化的策略来应对这一挑战,但随着底层动力学变得高维、混合或接触依赖,这种泛化变得越来越困难。为了解决这个问题,物理信息GCRL(Pi-GCRL)将最优控制启发的归纳偏置引入目标条件价值学习。虽然Pi-GCRL方法在导航和无目标到达领域已被证明有效,但它们在接触丰富任务中的可靠性仍不清楚,其中接触交互导致混合动力学、模式依赖的可控性和非光滑价值景观。在这项工作中,我们表明这些结构特性可能导致现有Pi-GCRL方法在朴素应用于接触丰富操作时性能下降。受此分析启发,我们引入了接触感知和分层公式,选择性地将物理信息归纳偏置应用于操作问题。我们的结果为将Pi-GCRL扩展到接触丰富操作提供了原则性的一步。

英文摘要:

Learning to reach arbitrary goals from sparse feedback requires agents to infer a rich notion of reachability across state--goal pairs. Goal-conditioned reinforcement learning (GCRL) tackles this challenge by learning policies that generalize across goals, but this generalization becomes increasingly difficult as the underlying dynamics become high-dimensional, hybrid, or contact-dependent. To address this issue, physics-informed GCRL (Pi-GCRL) introduces optimal-control-inspired inductive biases into goal-conditioned value learning. While Pi-GCRL methods have proven effective in navigation and object-free goal-reaching domains, their reliability in contact-rich tasks remains unclear, where contact interactions induce hybrid dynamics, mode-dependent controllability, and nonsmooth value landscapes. In this work, we show that these structural properties can cause existing Pi-GCRL methods to degrade when applied naively to contact-rich manipulation. Motivated by this analysis, we introduce contact-aware and hierarchical formulations that apply physics-informed inductive biases selectively across the manipulation problem. Our results provide a principled step toward extending Pi-GCRL to contact-rich manipulation.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑