SkyDreamer：基于模型强化学习的可解释端到端视觉无人机竞速
SkyDreamer: Interpretable End-to-End Vision-Based Drone Racing with Model-Based Reinforcement Learning
AI总结：
SkyDreamer是首个端到端视觉无人机竞速策略，利用基于模型的强化学习实现可解释的隐式状态估计，无需外部标定即可机载高速飞行，速度达21米/秒，并鲁棒应对视觉和电池等现实差距。
摘要：
自主无人机竞速（ADR）系统近期已达到冠军级性能，但仍高度专门化于无人机竞速场景。尽管端到端视觉方法有望实现更广泛的适用性，但迄今为止尚无系统能同时实现完整的仿真到现实迁移、机载执行和冠军级性能。在本工作中，我们提出了SkyDreamer，据我们所知，这是首个从像素级表示直接映射到电机命令的端到端视觉ADR策略。SkyDreamer基于informed Dreamer，这是一种基于模型的强化学习方法，其中世界模型解码仅在训练期间可用的特权信息。通过将此概念扩展到端到端视觉ADR，世界模型有效地充当隐式状态和参数估计器，大大提高了可解释性。SkyDreamer完全在机载运行，无需外部辅助，通过利用从世界模型隐藏状态解码出的状态来跟踪进度，从而解决视觉模糊性问题，并且无需外部相机标定，使得无需重新训练即可在不同无人机间快速部署。真实世界实验表明，SkyDreamer实现了鲁棒的高速飞行，可执行诸如倒飞筋斗、破S和梯子等紧凑机动，速度最高达21米/秒，加速度最高达6g。它还通过处理低质量分割掩码展示了非平凡的视觉仿真到现实迁移，并通过准确估计最大可达电机转速并实时调整飞行路径，展现出对电池电量耗尽的鲁棒性。这些结果突显了SkyDreamer对现实差距重要方面的适应性，在实现极高速度、敏捷飞行的同时带来了鲁棒性。
英文摘要：
Autonomous drone racing (ADR) systems have recently achieved champion-level performance, yet remain highly specific to drone racing. While end-to-end vision-based methods promise broader applicability, no system to date simultaneously achieves full sim-to-real transfer, onboard execution, and champion-level performance. In this work, we present SkyDreamer, to the best of our knowledge, the first end-to-end vision-based ADR policy that maps directly from pixel-level representations to motor commands. SkyDreamer builds on informed Dreamer, a model-based reinforcement learning approach where the world model decodes to privileged information only available during training. By extending this concept to end-to-end vision-based ADR, the world model effectively functions as an implicit state and parameter estimator, greatly improving interpretability. SkyDreamer runs fully onboard without external aid, resolves visual ambiguities by tracking progress using the state decoded from the world model's hidden state, and requires no extrinsic camera calibration, enabling rapid deployment across different drones without retraining. Real-world experiments show that SkyDreamer achieves robust, high-speed flight, executing tight maneuvers such as an inverted loop, a split-S and a ladder, reaching speeds of up to 21 m/s and accelerations of up to 6 g. It further demonstrates a non-trivial visual sim-to-real transfer by operating on poor-quality segmentation masks, and exhibits robustness to battery depletion by accurately estimating the maximum attainable motor RPM and adjusting its flight path in real-time. These results highlight SkyDreamer's adaptability to important aspects of the reality gap, bringing robustness while still achieving extremely high-speed, agile flight.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
