arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2410.04553cs.LGcs.SYeess.SY
收藏

用于模型预测控制的互模拟度量

Bisimulation metric for Model Predictive Control

University of California, Berkeley(加利福尼亚大学伯克利分校)

Yutaka Shimizu, Masayoshi Tomizuka

AI总结:

针对基于模型的强化学习现有方法在训练稳定性、噪声鲁棒性和计算效率上的不足,提出融入互模拟度量损失直接优化编码器的BS-MPC方法,在DeepMind Control Suite任务上表现优于现有基线。

摘要:

基于模型的强化学习在提升复杂环境中的样本效率和决策能力方面已展现出潜力。然而,现有方法在训练稳定性、噪声鲁棒性和计算效率方面面临挑战。本文提出了用于模型预测控制的互模拟度量方法(Bisimulation Metric for Model Predictive Control,BS-MPC),这是一种在目标函数中纳入互模拟度量损失以直接优化编码器的新方法。这种逐时间步的直接优化使学习到的编码器能够从原始状态空间中提取本质信息,同时舍弃无关细节,并防止梯度和误差发散。BS-MPC通过缩短训练时间,提升了训练稳定性、对输入噪声的鲁棒性以及计算效率。我们在DeepMind Control Suite的连续控制任务和基于图像的任务上对BS-MPC进行了评估,结果表明其性能和鲁棒性均优于当前最优的基线方法。

英文摘要:

Model-based reinforcement learning has shown promise for improving sample efficiency and decision-making in complex environments. However, existing methods face challenges in training stability, robustness to noise, and computational efficiency. In this paper, we propose Bisimulation Metric for Model Predictive Control (BS-MPC), a novel approach that incorporates bisimulation metric loss in its objective function to directly optimize the encoder. This time-step-wise direct optimization enables the learned encoder to extract intrinsic information from the original state space while discarding irrelevant details and preventing the gradients and errors from diverging. BS-MPC improves training stability, robustness against input noise, and computational efficiency by reducing training time. We evaluate BS-MPC on both continuous control and image-based tasks from the DeepMind Control Suite, demonstrating superior performance and robustness compared to state-of-the-art baseline methods.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑