arXiv 2410.223082026-01-14cs.RO

环境作为策略：在未见过的赛道上学习赛车

Environment as Policy: Learning to Race in Unseen Tracks

Robotics and Perception Group, Department of Informatics, University of Zurich（机器人感知组，信息学院，苏黎世大学）；Department of Neuroinformatics, University of Zurich and ETH Zurich（神经信息学院，苏黎世大学和苏黎世联邦理工学院）

Hongze Wang, Jiaxu Xing, Nico Messikommer, Davide Scaramuzza

备注：: Accepted at IEEE International Conference on Robotics and Automation (ICRA), 2025

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

本文提出了一种自适应环境塑造框架，使无人机能够在未见过的赛道上高效学习赛车，通过动态调整训练环境提升泛化能力。

摘要：

强化学习（RL）在复杂的机器人控制任务中取得了显著成功，例如无人机赛车，在已知的赛车赛道上，RL智能体已能超越人类冠军。然而，这些智能体在未见过的赛道配置中表现不佳，总是需要在新的赛道布局面前重新训练。本工作旨在开发能够有效泛化到新赛道配置的RL智能体，而无需重新训练。直接在多样化的赛道布局上训练的简单解决方案会加重智能体的负担，导致由于环境复杂性增加而影响智能体学习飞行的能力。为了提高RL智能体的泛化能力，我们提出了一种自适应的环境塑造框架，该框架根据智能体的表现动态调整训练环境。我们通过利用一个次级RL策略来设计环境，使环境在具有挑战性和可实现性之间达到平衡，使智能体能够逐步适应和改进。使用我们的自适应环境塑造方法，一个单一的赛车策略能够高效地在多样化的具有挑战性的赛道上学习赛车。在仿真和现实世界中的实验结果验证了我们的方法使无人机能够成功飞行复杂的和未见过的赛车赛道，优于现有的环境塑造技术。项目页面：http://rpg.ifi.uzh.ch/env_as_policy

英文摘要：

Reinforcement learning (RL) has achieved outstanding success in complex robot control tasks, such as drone racing, where the RL agents have outperformed human champions in a known racing track. However, these agents fail in unseen track configurations, always requiring complete retraining when presented with new track layouts. This work aims to develop RL agents that generalize effectively to novel track configurations without retraining. The naive solution of training directly on a diverse set of track layouts can overburden the agent, resulting in suboptimal policy learning as the increased complexity of the environment impairs the agent's ability to learn to fly. To enhance the generalizability of the RL agent, we propose an adaptive environment-shaping framework that dynamically adjusts the training environment based on the agent's performance. We achieve this by leveraging a secondary RL policy to design environments that strike a balance between being challenging and achievable, allowing the agent to adapt and improve progressively. Using our adaptive environment shaping, one single racing policy efficiently learns to race in diverse challenging tracks. Experimental results validated in both simulation and the real world show that our method enables drones to successfully fly complex and unseen race tracks, outperforming existing environment-shaping techniques. Project page: http://rpg.ifi.uzh.ch/env_as_policy.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。