arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2303.01173cs.ROcs.AIcs.LG
收藏

基于强化学习的氦原子资源受限位置保持算法

Resource-Constrained Station-Keeping for Helium Balloons using Reinforcement Learning

University of Bath(巴斯大学);Lux Aerobot

Jack Saunders, Loïc Prenevost, Özgür Şimşek, Alan Hunter, Wenbin Li

摘要:

高空气球已被证明可用于生态航空调查、大气监测和通信中继。然而,由于重量和功率的限制,有必要研究替代的推进模式,以在平流层中导航。最近,已经提出了强化学习作为一种控制方案,以保持气球在一个固定位置的区域,通过不同的相反的风场在不同的高度促进。虽然已经探索了基于气泵的位置保持,但是没有研究用于通风和压载致动气球的控制问题,这通常被用作低成本的替代方案。我们展示了如何将强化学习用于这种类型的气球。具体来说,我们使用软演员评论家算法,平均能够保持在50公里内的25%的飞行,符合国家的最先进的。此外,我们表明,所提出的控制器有效地减少了资源的消耗,从而支持长时间的飞行。我们框架的控制器作为一个连续的控制强化学习问题,它允许更多样化的轨迹范围,而不是目前的最先进的工作,它使用离散的动作空间。此外,通过连续控制,我们可以利用更大的上升速率,这是不可能使用空气泵。所需的上升速率被解耦成所需的高度和时间因素,以提供一个更透明的政策,相比,在以前的作品中使用的低级别的控制命令。最后,通过应用运动方程,我们建立了适当的阈值通风和压载,以防止代理人利用环境。更具体地说,我们通过对通风和压载实施限制,确保行动在物理上可行。

英文摘要:

High altitude balloons have proved useful for ecological aerial surveys, atmospheric monitoring, and communication relays. However, due to weight and power constraints, there is a need to investigate alternate modes of propulsion to navigate in the stratosphere. Very recently, reinforcement learning has been proposed as a control scheme to maintain the balloon in the region of a fixed location, facilitated through diverse opposing wind-fields at different altitudes. Although air-pump based station keeping has been explored, there is no research on the control problem for venting and ballasting actuated balloons, which is commonly used as a low-cost alternative. We show how reinforcement learning can be used for this type of balloon. Specifically, we use the soft actor-critic algorithm, which on average is able to station-keep within 50\;km for 25\% of the flight, consistent with state-of-the-art. Furthermore, we show that the proposed controller effectively minimises the consumption of resources, thereby supporting long duration flights. We frame the controller as a continuous control reinforcement learning problem, which allows for a more diverse range of trajectories, as opposed to current state-of-the-art work, which uses discrete action spaces. Furthermore, through continuous control, we can make use of larger ascent rates which are not possible using air-pumps. The desired ascent-rate is decoupled into desired altitude and time-factor to provide a more transparent policy, compared to low-level control commands used in previous works. Finally, by applying the equations of motion, we establish appropriate thresholds for venting and ballasting to prevent the agent from exploiting the environment. More specifically, we ensure actions are physically feasible by enforcing constraints on venting and ballasting.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑