大型语言模型作为双人博弈中的智能体
Large Language Models as Agents in Two-Player Games
AI总结：
本文将大型语言模型的训练过程统一为单一机器学习范式，提出将其重新概念化为双人博弈中的智能体学习，从而揭示对齐问题的新视角并启发新型数据准备与训练技术。
摘要：
通过将大型语言模型（LLMs）的训练过程（通常包括预训练、监督微调以及基于人类反馈的强化学习）在单一且统一的机器学习范式中进行正式定义，我们能够获得推进 LLM 技术的关键见解。这篇立场论文阐明了 LLMs 的训练方法与双人博弈中智能体开发所采用的策略之间的相似性，这些策略在博弈论、强化学习和多智能体系统中均有研究。我们提出从基于语言的博弈中智能体学习的角度重新概念化 LLM 的学习过程。该框架揭示了 LLM 开发中的成功与挑战的新视角，为解决对齐问题及其他战略考量提供了新的理解。此外，我们的双人博弈方法为训练 LLMs 的新型数据准备和机器学习技术提供了启示。
英文摘要：
By formally defining the training processes of large language models (LLMs), which usually encompasses pre-training, supervised fine-tuning, and reinforcement learning with human feedback, within a single and unified machine learning paradigm, we can glean pivotal insights for advancing LLM technologies. This position paper delineates the parallels between the training methods of LLMs and the strategies employed for the development of agents in two-player games, as studied in game theory, reinforcement learning, and multi-agent systems. We propose a re-conceptualization of LLM learning processes in terms of agent learning in language-based games. This framework unveils innovative perspectives on the successes and challenges in LLM development, offering a fresh understanding of addressing alignment issues among other strategic considerations. Furthermore, our two-player game approach sheds light on novel data preparation and machine learning techniques for training LLMs.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
