面向非线性控制的收缩感知强化学习：具备统计鲁棒性
Contraction-Aware Reinforcement Learning for Nonlinear Control with Statistical Robustness
AI总结：
本研究针对非线性路径跟踪问题中控制收缩度量（CCM）策略的长期最优性与鲁棒性缺陷，提出收缩感知强化学习（CARL）算法，将CCM与RL结合，经模拟和真实机器人实验验证其性能与鲁棒性。
摘要：
控制收缩度量（CCMs）由黎曼度量定义，在该度量下闭环系统是增量指数稳定的，为非线性路径跟踪问题中合成收缩策略提供了可构造框架。然而，合成策略虽能保证逐点满足CCM条件，却无法确保在瞬态和稳态区域内的长期最优性（即最小化累积轨迹级跟踪误差）；此外，当用近似动力学构建CCM时，这些策略的短视特性还可能使其更易受学习偏差影响。为解决上述问题，我们提出将CCMs与强化学习（RL）相结合：CCM为学习策略提供动力学信息反馈，该策略具备稳定性保证（即收缩感知），而RL则为在近似动力学下最小化累积跟踪误差提供框架。给定预训练动力学模型，我们的算法收缩感知强化学习（CARL）可同时学习生成CCMs并针对由这些CCM定义的奖励优化策略。我们在模拟和真实机器人实验中，与相关基线对比验证了CARL可提升路径跟踪性能，且对近似动力学的误差具备鲁棒性；同时，我们也为将CCMs集成到RL中提供了理论依据。我们的代码可在指定网址获取，真实机器人实验的视频也可在指定网址查看。
英文摘要：
Control contraction metrics (CCMs)-defined by Riemannian metrics under which a closed-loop system is incrementally exponentially stable-offer a constructive framework for synthesizing contracting policies in nonlinear path-tracking problems. However, while the synthesized policies ensure pointwise satisfaction of the CCM conditions, they may not ensure long-term optimality (i.e., minimizing cumulative trajectory-level tracking error) over both transient and steady-state regimes. Furthermore, the myopic nature of these policies could also make them more susceptible to learning biases when approximate dynamics are used to formulate CCMs. To address these issues, we propose to integrate CCMs into reinforcement learning (RL). CCMs provide dynamics-informed feedback for learning a policy that has a stability guarantee-i.e., is contraction-aware-while RL provides a framework for minimizing cumulative tracking error under approximate dynamics. Given a pretrained dynamics model, our algorithm, contraction-aware RL (CARL), simultaneously learns to generate CCMs and optimize a policy for rewards defined by those CCMs. We demonstrate that CARL enhances path-tracking performance and is robust to errors in approximated dynamics compared to relevant baselines in both simulated and real-world robot experiments. We also provide theoretical rationale for integrating CCMs into RL. Our code is available at https://github.com/Mgineer117/CARL, and a video of our real-world robot experiments can be found at https://youtu.be/sOJ4hulbop0.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
