玩文字游戏，用奖励改进：训练语言模型进行创意联想
Playing with Words, Improving with Rewards: Training Language Models for Creative Association
AI总结：
本研究通过强化学习与可验证奖励（RLVR）在Codenames游戏上训练LLM，探索了规模依赖的精确度-创造力权衡，发现8B模型在保持推理能力的同时提升创造力，而小模型则牺牲创造力换取推理精度。
摘要：
大型语言模型（LLM）正被应用于日益困难的问题和用例。为了有效导航其广阔的解决方案空间，LLM需要具备创造力。然而，创造力的主观性和人类判断的局限性使得训练LLM的创造力尤其具有挑战性。作为解决方案，我们在Codenames（一个词联想游戏）上训练LLM，该游戏锻炼了创造力的两个核心轴——发散思维和收敛思维，同时产生客观可验证的结果。这种可验证性使我们能够绕过人类判断，并使用具有可验证奖励的强化学习（RLVR）进行训练。我们训练了Qwen3-1.7B、4B和8B模型，并在十个创造力和四个推理基准上评估它们。我们发现精确度-创造力权衡是规模依赖的：8B模型优先考虑创造力而非精确度，而1.7B和4B模型则以牺牲创造力为代价获得推理精确度。具体来说，8B模型在8个创造力基准上显示出适度但一致的提升，且推理能力仅略有下降，而较小的模型在推理任务上取得了显著提升。我们的研究提出了一种可扩展且有效的解决方案来训练LLM的创造力。
英文摘要：
Large Language Models (LLMs) are being applied to increasingly difficult problems and use cases. To navigate their vast solution spaces effectively, LLMs need to be creative. Yet the subjective nature of creativity and the limits of human judgment make training LLMs for creativity especially challenging. As a solution, we train LLMs on Codenames, a word-association game that exercises the two central axes of creativity, divergent and convergent thinking, while yielding objectively verifiable outcomes. This verifiability lets us bypass human judgment and train with Reinforcement Learning with Verifiable Rewards (RLVR). We train Qwen3-1.7B, 4B, and 8B models and evaluate them on ten creativity and four reasoning benchmarks. We find that the precision-diversity trade-off is scale-dependent: the 8B model prioritizes creativity over precision, while the 1.7B and 4B models gain reasoning precision at the cost of creativity. Concretely, the 8B model shows modest but consistent creativity gains (8 of 10 benchmarks) with only minor reasoning degradation, whereas the smaller models achieve substantial gains on reasoning tasks. Our study presents a scalable and effective solution to train LLMs for creativity.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
