arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2408.13328cs.LGcs.AI
收藏

战斗模拟中基于分段线性空间衰减的局部观测抽象在强化学习应用

Localized Observation Abstraction Using Piecewise Linear Spatial Decay for Reinforcement Learning in Combat Simulations

Naval Postgraduate School(海军研究生院)

Scotty Black, Christian Darken

AI总结:

针对战斗模拟中RL训练效率随复杂度指数下降的问题,提出分段线性空间衰减的局部观测抽象方法,简化状态空间并保留关键信息,显著提升训练效率且优于全局观测。

摘要:

在战斗模拟领域,,由于此类环境的动态性和复杂性,深度强化学习(RL)智能体的训练和部署仍面临重大挑战。不幸的是,随着场景复杂性和可用信息的增加,达到特定性能阈值所需的训练时间不仅会增加,而且通常呈指数级增长。这种关系凸显了复杂性在训练RL智能体方面的深远影响。本文提出了一种新方法,以解决使用RL训练人工智能(AI)智能体时的这一局限性。已证明传统RL方法由于现实世界的计算约束和RL已知的样本低效挑战,难以在这些高维动态环境中发挥作用。为克服这些局限性,我们提出了一种使用分段线性空间衰减的局部观测抽象方法。该技术简化了状态空间,降低了计算需求,同时保留了关键信息,从而在空间关系通常至关重要的动态环境中提高了AI训练效率。我们的分析表明,随着场景复杂度的增加,这种局部观测方法始终优于更传统的全局观测方法。本文推进了RL观测抽象的研究,说明了具有分段线性空间衰减的局部观测如何为动态环境中的大状态表示挑战提供有效解决方案。

英文摘要:

In the domain of combat simulations, the training and deployment of deep reinforcement learning (RL) agents still face substantial challenges due to the dynamic and intricate nature of such environments. Unfortunately, as the complexity of the scenarios and available information increases, the training time required to achieve a certain threshold of performance does not just increase, but often does so exponentially. This relationship underscores the profound impact of complexity in training RL agents. This paper introduces a novel approach that addresses this limitation in training artificial intelligence (AI) agents using RL. Traditional RL methods have been shown to struggle in these high-dimensional, dynamic environments due to real-world computational constraints and the known sample inefficiency challenges of RL. To overcome these limitations, we propose a method of localized observation abstraction using piecewise linear spatial decay. This technique simplifies the state space, reducing computational demands while still preserving essential information, thereby enhancing AI training efficiency in dynamic environments where spatial relationships are often critical. Our analysis reveals that this localized observation approach consistently outperforms the more traditional global observation approach across increasing scenario complexity levels. This paper advances the research on observation abstractions for RL, illustrating how localized observation with piecewise linear spatial decay can provide an effective solution to large state representation challenges in dynamic environments.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑