arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2411.04799cs.CLcs.AI
收藏

Kwai-STaR:将 LLMs 转化为状态转移推理器

Kwai-STaR: Transform LLMs into State-Transition Reasoners

Shenzhen International Graduate School, Tsinghua University(清华大学深圳国际研究生院);Kuaishou Technology(快手科技);Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所);Pengcheng Laboratory(鹏城实验室)
Xingyu Lu, Yuhang Hu, Changyi Liu, Tianke Zhang, Zhenyu Yang, Zhixiang Ding, Shengsheng Qian, Meng Du, Ruiwen Kang, Kaiyu Tang, Fan Yang, Tingting Gao, Di Zhang… 展开作者

Xingyu Lu, Yuhang Hu, Changyi Liu, Tianke Zhang, Zhenyu Yang, Zhixiang Ding, Shengsheng Qian, Meng Du, Ruiwen Kang, Kaiyu Tang, Fan Yang, Tingting Gao, Di Zhang, Hai-Tao Zheng, Bin Wen

AI总结:

本文提出 Kwai-STaR 框架,将数学问题求解建模为状态转移过程,通过定义状态空间、生成转移数据及课程训练策略,把 LLMs 转化为状态转移推理器,有效提升了 Mistral-7B 和 LLaMA-3 在 GSM8K 等数据集上的数学推理性能与效率。

摘要:

数学推理对 LLMs 的认知能力构成了重大挑战。已有多种方法被提出以增强 LLMs 的数学能力。然而,很少有人认识到状态转移对 LLM 推理的价值。在本研究中,我们将数学问题求解定义为从初始未解决状态转移到最终已解决状态的过程,并提出 Kwai-STaR 框架,该框架将 LLMs 转化为状态转移推理器(State-Transition Reasoners),以提升其直观推理能力。我们的方法包含三个主要步骤:(1)定义针对数学推理的状态空间。(2)基于该状态空间生成状态转移数据。(3)通过课程训练策略将原始 LLMs 转化为状态转移推理器。我们的实验验证了 Kwai-STaR 在增强数学推理方面的有效性:在小规模的 Kwai-STaR 数据集上训练后,包括 Mistral-7B 和 LLaMA-3 在内的通用 LLMs 在 GSM8K 和 GSM-Hard 数据集上取得了显著的性能提升。此外,基于状态转移的设计赋予了 Kwai-STaR 卓越的训练和推理效率。进一步的实验正在进行中,以确立 Kwai-STaR 的通用性。

英文摘要:

Mathematical reasoning presents a significant challenge to the cognitive capabilities of LLMs. Various methods have been proposed to enhance the mathematical ability of LLMs. However, few recognize the value of state transition for LLM reasoning. In this work, we define mathematical problem-solving as a process of transiting from an initial unsolved state to the final resolved state, and propose Kwai-STaR framework, which transforms LLMs into State-Transition Reasoners to improve their intuitive reasoning capabilities. Our approach comprises three main steps: (1) Define the state space tailored to the mathematical reasoning. (2) Generate state-transition data based on the state space. (3) Convert original LLMs into State-Transition Reasoners via a curricular training strategy. Our experiments validate the effectiveness of Kwai-STaR in enhancing mathematical reasoning: After training on the small-scale Kwai-STaR dataset, general LLMs, including Mistral-7B and LLaMA-3, achieve considerable performance gain on the GSM8K and GSM-Hard dataset. Additionally, the state transition-based design endows Kwai-STaR with remarkable training and inference efficiency. Further experiments are underway to establish the generality of Kwai-STaR.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑