面向工业波浪能转换器的多智能体强化学习控制器的跳跃训练方法
Skip Training for Multi-Agent Reinforcement Learning Controller for Industrial Wave Energy Converters
AI总结:
针对工业波浪能转换器传统控制器的局限及多智能体强化学习训练难收敛问题,提出跳跃训练和混合训练初始化方法,提升了发电效率。
摘要:
新型波浪能转换器(WEC)配备多支腿和多发电机以最大化发电量。传统控制器在捕捉复杂波浪模式方面存在局限性,且必须高效提升能量捕获效率。本文提出一种多智能体强化学习(MARL)控制器,其性能优于传统使用的弹簧阻尼器控制器。我们的初步研究表明,问题的复杂性导致训练难以收敛。为此,我们提出一种新颖的跳跃训练方法,与默认的MARL训练相比,该方法能让MARL训练克服性能饱和,收敛到更优的控制器,提升发电量。我们还提出另一种新颖的混合训练初始化(STHTI)方法:MARL控制器的单个智能体可先分别针对基线弹簧阻尼器(SD)控制器进行初始训练,后续迭代中可逐个训练单个智能体或同时训练所有智能体,以加速收敛。我们采用异步优势演员-评论家(A3C)算法,所提出的MARL控制器相比基线弹簧阻尼器控制器实现了两位数的能源效率提升。
英文摘要:
Recent Wave Energy Converters (WEC) are equipped with multiple legs and generators to maximize energy generation. Traditional controllers have shown limitations to capture complex wave patterns and the controllers must efficiently maximize the energy capture. This paper introduces a Multi-Agent Reinforcement Learning controller (MARL), which outperforms the traditionally used spring damper controller. Our initial studies show that the complex nature of problems makes it hard for training to converge. Hence, we propose a novel skip training approach which enables the MARL training to overcome performance saturation and converge to more optimum controllers compared to default MARL training, boosting power generation. We also present another novel hybrid training initialization (STHTI) approach, where the individual agents of the MARL controllers can be initially trained against the baseline Spring Damper (SD) controller individually and then be trained one agent at a time or all together in future iterations to accelerate convergence. We achieved double-digit gains in energy efficiency over the baseline Spring Damper controller with the proposed MARL controllers using the Asynchronous Advantage Actor-Critic (A3C) algorithm.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。