基于自然语言命令的StarCraft II游戏状态的叙事引导强化学习
Grounding Natural Language Commands to StarCraft II Game States for Narration-Guided Reinforcement Learning
摘要：
虽然深度强化学习技术已经使智能体能够成功地学习执行许多以前无法学习的任务，但这些技术仍然容易受到长期存在的{ \em奖励稀疏}问题的影响。这对于训练智能体玩星际争霸II这样的任务来说尤其如此，星际争霸II是一款实时战略游戏，只有在游戏结束时才会给予奖励，而游戏通常很长。虽然这个问题可以通过奖励塑造来解决，但这种方法通常需要具有专业知识的人类专家。通过自然语言叙事的更容易接近的范式，使奖励塑造的愿景的启发下，我们调查在多大程度上，我们可以通过接地这些叙事的具体目标的状态。我们提出了一种使用多输入深度神经网络的相互嵌入模型，该模型将一系列自然语言命令投影到与相应目标状态相同的高维表示空间中。我们表明，使用这个模型，我们可以学习一个嵌入空间与可分离的和不同的集群，准确地映射自然语言命令到相应的游戏状态。我们还讨论了这个模型如何允许使用叙述作为一个强大的形式的奖励塑造，以提高RL的性能和效率。
英文摘要：
While deep reinforcement learning techniques have led to agents that are successfully able to learn to perform a number of tasks that had been previously unlearnable, these techniques are still susceptible to the longstanding problem of {\em reward sparsity}. This is especially true for tasks such as training an agent to play StarCraft II, a real-time strategy game where reward is only given at the end of a game which is usually very long. While this problem can be addressed through reward shaping, such approaches typically require a human expert with specialized knowledge. Inspired by the vision of enabling reward shaping through the more-accessible paradigm of natural-language narration, we investigate to what extent we can contextualize these narrations by grounding them to the goal-specific states. We present a mutual-embedding model using a multi-input deep-neural network that projects a sequence of natural language commands into the same high-dimensional representation space as corresponding goal states. We show that using this model we can learn an embedding space with separable and distinct clusters that accurately maps natural-language commands to corresponding game states . We also discuss how this model can allow for the use of narrations as a robust form of reward shaping to improve RL performance and efficiency.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
