arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2506.00577cs.AIcs.CLcs.GTcs.MA
收藏

像经济学家一样推理:在经济问题上的后训练诱导大语言模型的策略泛化

Reasoning Like an Economist: Post-Training on Economic Problems Induces Strategic Generalization in LLMs

Duke University(杜克大学);EPIC Lab, Shanghai Jiao Tong University(上海交通大学EPIC实验室);Qwen Team, Alibaba Group(阿里集团Qwen团队);University of Pennsylvania(宾夕法尼亚大学);The University of Chicago(芝加哥大学)

Yufa Zhou, Shaobo Wang, Xingyu Dong, Xiangqi Jin, Yifang Chen, Yue Min, Kexin Yang, Xingzhang Ren, Dayiheng Liu, Linfeng Zhang

AI总结:

本文提出在2100个经济推理问题上对7B开源LLM进行SFT和RLVR后训练,验证了该领域对齐方法能有效提升模型在多智能体场景下的结构化推理、经济合理性及策略泛化能力。

摘要:

由于复杂的奖励建模、动态的智能体交互和严苛的泛化要求,直接为多智能体系统(MAS)训练大语言模型(LLMs)仍然具有挑战性。本文探讨了后训练技术,特别是监督微调(SFT)和带可验证奖励的强化学习(RLVR),能否有效泛化到多智能体场景。我们将经济推理作为试验台,利用其在数学和博弈论方面的坚实基础、对结构化分析推理的需求,以及与市场设计、资源分配和政策分析等现实世界应用的相关性。我们引入了Recon(像经济学家一样推理),这是一个7B参数的开源LLM,在包含2,100个高质量经济推理问题的人工精选数据集上进行了后训练。在经济推理基准和多智能体博弈上的全面评估显示,结构化推理和经济合理性均有显著提升。这些结果强调了领域对齐后训练在增强推理和智能体对齐方面的潜力,并揭示了SFT和RL在塑造模型行为中的作用。代码可在 https://github.com/MasterZhou1/Recon 获取。

英文摘要:

Directly training Large Language Models (LLMs) for Multi-Agent Systems (MAS) remains challenging due to intricate reward modeling, dynamic agent interactions, and demanding generalization requirements. This paper explores whether post-training techniques, specifically Supervised Fine-Tuning (SFT) and Reinforcement Learning with Verifiable Rewards (RLVR), can effectively $\textit{generalize}$ to multi-agent scenarios. We use economic reasoning as a testbed, leveraging its strong foundations in mathematics and game theory, its demand for structured analytical reasoning, and its relevance to real-world applications such as market design, resource allocation, and policy analysis. We introduce $\textbf{Recon}$ ($\textbf{R}$easoning like an $\textbf{ECON}$omist), a 7B-parameter open-source LLM post-trained on a hand-curated dataset of 2,100 high-quality economic reasoning problems. Comprehensive evaluation on economic reasoning benchmarks and multi-agent games reveals clear improvements in structured reasoning and economic rationality. These results underscore the promise of domain-aligned post-training for enhancing reasoning and agent alignment, shedding light on the roles of SFT and RL in shaping model behavior. Code is available at https://github.com/MasterZhou1/Recon .

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑