使用多智能体强化学习的缆索悬挂负载分散式空中操作
Decentralized Aerial Manipulation of a Cable-Suspended Load using Multi-Agent Reinforcement Learning
AI总结：
本文提出首个基于多智能体强化学习的分散式方法，使微型飞行器团队无需全局状态和通信即可实现缆索悬挂负载的6自由度操作，新动作空间设计保障了从仿真到现实的可靠迁移，实验证明其跟踪性能与集中式方法相当且具备高鲁棒性。
摘要：
本文提出了首个分散式方法，利用微型飞行器（MAV）团队实现真实世界中缆索悬挂负载的6自由度（6-DoF）操作。该方法利用多智能体强化学习（MARL）为每个MAV训练外环控制策略。与利用集中式方案的最先进控制器不同，该策略不需要全局状态、MAV间通信，也不需要邻近MAV信息。相反，智能体仅通过负载位姿观测进行隐式通信，从而实现了高可扩展性和灵活性。这也显著降低了推理期间的算力成本，使得策略能够进行机载部署。此外，我们引入了一种使用线加速度和机体角速度的MAV新动作空间设计。该选择结合鲁棒的底层控制器，尽管动态3D运动期间缆索张力引起了显著不确定性，仍实现了可靠的从仿真到现实（sim-to-real）的迁移。我们在各种真实世界实验中验证了该方法，包括负载模型不确定性下的全位姿控制，展示了与最先进集中式方法相当的设定点跟踪性能。我们还演示了具有异构控制策略的智能体间的协作，以及对一架MAV在飞行中完全失效的鲁棒性。
英文摘要：
This paper presents the first decentralized method to enable real-world 6-DoF manipulation of a cable-suspended load using a team of Micro-Aerial Vehicles (MAVs). Our method leverages multi-agent reinforcement learning (MARL) to train an outer-loop control policy for each MAV. Unlike state-of-the-art controllers that utilize a centralized scheme, our policy does not require global states, inter-MAV communications, nor neighboring MAV information. Instead, agents communicate implicitly through load pose observations alone, which enables high scalability and flexibility. It also significantly reduces computing costs during inference time, enabling onboard deployment of the policy. In addition, we introduce a new action space design for the MAVs using linear acceleration and body rates. This choice, combined with a robust low-level controller, enables reliable sim-to-real transfer despite significant uncertainties caused by cable tension during dynamic 3D motion. We validate our method in various real-world experiments, including full-pose control under load model uncertainties, showing setpoint tracking performance comparable to the state-of-the-art centralized method. We also demonstrate cooperation amongst agents with heterogeneous control policies, and robustness to the complete in-flight loss of one MAV. Videos of experiments: https://autonomousrobots.nl/paper_websites/aerial-manipulation-marl
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
