FIREBALL:一个带有结构化游戏状态信息的《龙与地下城》实况游戏数据集
FIREBALL: A Dataset of Dungeons and Dragons Actual-Play with Structured Game State Information
AI总结:
本文提出FIREBALL数据集,收录近25,000个带真实游戏状态的D&D实况会话,证明利用Avrae状态信息可提升LLM的游戏回合生成质量,并展示LLM经微调后能生成可执行的Avrae命令。
摘要:
《龙与地下城》(D&D)是一款桌面角色扮演游戏,具有玩家之间复杂的自然语言交互和隐藏状态信息。最近的研究表明,能够访问状态信息的大型语言模型(LLMs)比仅使用对话历史的大型语言模型能够生成更高质量的游戏回合。然而,先前的工作使用的游戏状态信息是启发式创建的,并非真正的黄金标准游戏状态。我们提出了FIREBALL,这是一个大型数据集,包含来自Discord上真实D&D游戏过程的近25,000个独立会话,并带有真实的游戏状态信息。我们记录了使用Avrae机器人的玩家的游戏过程会话,该机器人是为帮助人们在线上玩D&D而开发的,捕获了语言、游戏命令和底层游戏状态信息。我们证明,FIREBALL可以通过使用Avrae状态信息来改进自然语言生成(NLG),同时提升自动评估指标和人类质量判断。此外,我们还表明,大型语言模型可以生成可执行的Avrae命令,尤其是在经过微调之后。
英文摘要:
Dungeons & Dragons (D&D) is a tabletop roleplaying game with complex natural language interactions between players and hidden state information. Recent work has shown that large language models (LLMs) that have access to state information can generate higher quality game turns than LLMs that use dialog history alone. However, previous work used game state information that was heuristically created and was not a true gold standard game state. We present FIREBALL, a large dataset containing nearly 25,000 unique sessions from real D&D gameplay on Discord with true game state info. We recorded game play sessions of players who used the Avrae bot, which was developed to aid people in playing D&D online, capturing language, game commands and underlying game state information. We demonstrate that FIREBALL can improve natural language generation (NLG) by using Avrae state information, improving both automated metrics and human judgments of quality. Additionally, we show that LLMs can generate executable Avrae commands, particularly after finetuning.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。