通过衰减残差策略优化实现高效的现实世界自主赛车
Efficient Real-World Autonomous Racing via Attenuated Residual Policy Optimization
AI总结：
本文提出α-RPO方法，通过逐步衰减基础策略生成独立神经策略，减少系统复杂度并提升驾驶性能，适用于现实世界机器人部署。
摘要：
残差策略学习（RPL）通过深度强化学习（DRL）使学习策略细化静态基础策略，在各种机器人应用中表现出色。其效果在自主赛车领域尤为明显，该领域是现实世界DRL的挑战性基准。然而，部署基于RPL的控制器会引入系统复杂性和推理延迟。我们通过引入一种名为衰减残差策略优化（α-RPO）的RPL扩展方法来解决这一问题。与标准RPL不同，α-RPO通过逐步衰减基础策略生成独立的神经策略，最初用于引导学习。此外，这种机制使基础策略能够使用在最终部署中不需要的传感器模态，实现一种特权学习。我们设计α-RPO以无缝集成PPO，确保基础控制器的衰减影响在策略优化过程中动态补偿。我们通过构建一个1:10比例的自主赛车框架来评估α-RPO。在模拟和零样本现实世界迁移至Roboracer汽车的测试中，α-RPO不仅减少了系统复杂度，还比基线方法提高了驾驶性能，展示了其在机器人部署中的实用性。我们的代码可在https://github.com/raphajaner/arpo_racing获取。
英文摘要：
Residual policy learning (RPL), in which a learned policy refines a static base policy using deep reinforcement learning (DRL), has shown strong performance across various robotic applications. Its effectiveness is particularly evident in autonomous racing, a domain that serves as a challenging benchmark for real-world DRL. However, deploying RPL-based controllers introduces system complexity and increases inference latency. We address this by introducing an extension of RPL named attenuated residual policy optimization ($α$-RPO). Unlike standard RPL, $α$-RPO yields a standalone neural policy by progressively attenuating the base policy, which initially serves to bootstrap learning. Furthermore, this mechanism enables a form of privileged learning, where the base policy is permitted to use sensor modalities not required for final deployment. We design $α$-RPO to integrate seamlessly with PPO, ensuring that the attenuated influence of the base controller is dynamically compensated during policy optimization. We evaluate $α$-RPO by building a framework for 1:10-scaled autonomous racing around it. In both simulation and zero-shot real-world transfer to Roboracer cars, $α$-RPO not only reduces system complexity but also improves driving performance compared to baselines - demonstrating its practicality for robotic deployment. Our code is available at: https://github.com/raphajaner/arpo_racing.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
