通过可解释的Hamilton-Jacobi可达性引导扰动学习鲁棒策略
Learning Robust Policies via Interpretable Hamilton-Jacobi Reachability-Guided Disturbances
AI总结：
本文提出一种结合Hamilton-Jacobi可达性分析与对抗强化学习的鲁棒策略训练框架，利用可解释的最坏情况扰动替代黑盒对抗者，在到达-回避博弈和四旋翼稳定任务中验证了其有效性与一致性。
摘要：
深度强化学习（RL）在具有复杂和异质动态特性的机器人领域取得了显著成功。然而，其对未知扰动和对抗攻击的脆弱性仍然是一个重大挑战。本文提出了一种鲁棒策略训练框架，将基于模型的控制原理与对抗性RL训练相结合，以提高鲁棒性，而无需外部黑盒对抗者。我们的方法引入了一种新颖的Hamilton-Jacobi可达性引导扰动用于对抗性RL训练，其中我们使用可解释的最坏情况或近最坏情况扰动作为针对鲁棒策略的对抗者。我们在三个不同的任务上评估了其有效性：在仿真和真实环境中的到达-回避博弈，以及在仿真中的高动态四旋翼稳定任务。我们验证了学习到的critic网络与真实HJ值函数一致，同时策略网络表现出与其他基于学习的方法相当的性能。
英文摘要：
Deep Reinforcement Learning (RL) has shown remarkable success in robotics with complex and heterogeneous dynamics. However, its vulnerability to unknown disturbances and adversarial attacks remains a significant challenge. In this paper, we propose a robust policy training framework that integrates model-based control principles with adversarial RL training to improve robustness without the need for external black-box adversaries. Our approach introduces a novel Hamilton-Jacobi reachability-guided disturbance for adversarial RL training, where we use interpretable worst-case or near-worst-case disturbances as adversaries against the robust policy. We evaluated its effectiveness across three distinct tasks: a reach-avoid game in both simulation and real-world settings, and a highly dynamic quadrotor stabilization task in simulation. We validate that our learned critic network is consistent with the ground-truth HJ value function, while the policy network shows comparable performance with other learning-based methods.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
