SC-Phi2：面向《星际争霸II》宏观管理任务的微调小型语言模型
SC-Phi2: A Fine-tuned Small Language Model for StarCraft II Macromanagement Tasks
AI总结：
本文提出SC-Phi2，通过自监督微调Phi2并结合BLIP-2视觉Transformer与LoRA量化，在单GPU上实现《星际争霸II》宏观管理任务的高效预测。
摘要：
本文介绍了SC-Phi2，一个针对《星际争霸II》宏观管理任务进行微调的小型语言模型。Phi2、Gemma和DistilBERT等小型语言模型是大型语言模型（LLMs）的精简版本，参数更少，运行所需的算力和内存也更低。为了让微软的Phi2模型学习《星际争霸》相关知识，我们构建了一个新的SC2文本数据集，其中包含《星际争霸》种族、角色和行动的信息，并利用该数据集以自监督学习方式对Phi-2进行微调。我们将该语言模型与来自预训练BLIP-2（Bootstrapping Language Image Pre-training）模型的视觉Transformer（ViT）配对，并在MSC回放数据集上对其进行微调。这使我们能够构建包含视觉游戏状态信息的动态提示。与《星际争霸》LLMs中使用的大型模型（如GPT-3.5）不同，Phi2主要基于教科书数据训练，除我们训练过程所提供的信息外，几乎不包含《星际争霸II》的固有知识。通过使用LoRA（Low-rank Adaptation）和量化技术，我们的模型可以在单个GPU上完成训练。我们证明，该模型在建造顺序和全局状态预测等微观管理任务上表现良好，且仅使用少量参数。
英文摘要：
This paper introduces SC-Phi2, a fine-tuned StarCraft II small language model for macromanagement tasks. Small language models, like Phi2, Gemma, and DistilBERT, are streamlined versions of large language models (LLMs) with fewer parameters that require less power and memory to run. To teach Microsoft's Phi2 model about StarCraft, we create a new SC2 text dataset with information about StarCraft races, roles, and actions and use it to fine-tune Phi-2 with self-supervised learning. We pair this language model with a Vision Transformer (ViT) from the pre-trained BLIP-2 (Bootstrapping Language Image Pre-training) model, fine-tuning it on the MSC replay dataset. This enables us to construct dynamic prompts that include visual game state information. Unlike the large models used in StarCraft LLMs such as GPT-3.5, Phi2 is trained primarily on textbook data and contains little inherent knowledge of StarCraft II beyond what is provided by our training process. By using LoRA (Low-rank Adaptation) and quantization, our model can be trained on a single GPU. We demonstrate that our model performs well at micromanagement tasks such as build order and global state prediction with a small number of parameters.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
