混合专家在混合强化学习场景中的应用
Mixture of Experts in a Mixture of RL settings
AI总结：
本研究通过多任务强化学习设置考察混合专家（MoE）在非平稳环境下的作用，揭示其提升学习能力的机制，并指导在演员-评论家网络中集成MoE组件。
摘要：
混合专家（MoEs）因推理效率提升、对分布式训练的适应性及模块化特性，在（自）监督学习中日益受到重视。先前研究表明，MoEs可通过增加网络参数数量并减少休眠神经元，显著提升深度强化学习（DRL）性能，从而增强模型的学习能力及应对非平稳性的能力。本研究进一步阐明MoEs处理非平稳性的能力，并通过多任务训练在“放大”非平稳性的DRL设置中考察MoEs，为MoEs提升学习能力提供进一步证据。与先前工作相比，我们的多任务结果使我们能更深入地理解MoE在DRL训练中产生有益效果的根本原因、各MoE组件的影响，以及如何最佳地将它们整合进基于演员-评论家的DRL网络。最后，我们也确认了先前工作的结果。
英文摘要：
Mixtures of Experts (MoEs) have gained prominence in (self-)supervised learning due to their enhanced inference efficiency, adaptability to distributed training, and modularity. Previous research has illustrated that MoEs can significantly boost Deep Reinforcement Learning (DRL) performance by expanding the network's parameter count while reducing dormant neurons, thereby enhancing the model's learning capacity and ability to deal with non-stationarity. In this work, we shed more light on MoEs' ability to deal with non-stationarity and investigate MoEs in DRL settings with "amplified" non-stationarity via multi-task training, providing further evidence that MoEs improve learning capacity. In contrast to previous work, our multi-task results allow us to better understand the underlying causes for the beneficial effect of MoE in DRL training, the impact of the various MoE components, and insights into how best to incorporate them in actor-critic-based DRL networks. Finally, we also confirm results from previous work.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
