arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2501.01668cs.CL
收藏

基于CoT的合成器:通过答案合成提升大语言模型性能

CoT-based Synthesizer: Enhancing LLM Performance through Answer Synthesis

School of Information, Renmin University of China(中国人民大学信息学院);Tsinghua University(清华大学);Key Laboratory of Data Engineering and Knowledge Engineering, Beijing, China(数据工程与知识工程重点实验室)

Bohan Zhang, Xiaokang Zhang, Jing Zhang, Jifan Yu, Sijia Luo, Jie Tang

AI总结:

针对现有推理缩放方法依赖候选答案质量、全错时无法生成正确结果的问题,提出基于CoT的答案合成策略,搭配自动化数据生成流水线,用小模型训练即可提升大模型推理准确率,在MATH等数据集上取得显著增益。

摘要:

当前的推理缩放方法,如Self-consistency(自一致性)和Best-of-N(N中选优),已被证明能有效提升大语言模型(LLM)在复杂推理任务上的准确率。但这些方法高度依赖候选响应的质量,当所有候选答案均错误时无法生成正确结果。本文提出一种新型推理缩放策略——CoT-based Synthesizer(基于思维链的合成器),该策略利用CoT(思维链)推理,通过分析多个候选响应中的互补信息合成更优答案,即便所有候选响应都存在缺陷也能生效。为实现轻量且低成本的部署,我们引入了一套自动化数据生成流水线,可生成多样化的训练数据。这使得在该数据上训练的较小规模LLM,能够提升包括基于API的LLM在内的更大模型的推理准确率。在四个基准数据集、七个策略模型上的实验结果表明,我们的方法显著提升了性能:在MATH数据集上,Llama3-8B的性能提升11.8%,GPT-4o提升10.3%。相应的训练数据和代码已在https://github.com/RUCKBReasoning/CoT-based-Synthesizer 公开。

英文摘要:

Current inference scaling methods, such as Self-consistency and Best-of-N, have proven effective in improving the accuracy of LLMs on complex reasoning tasks. However, these methods rely heavily on the quality of candidate responses and are unable to produce correct answers when all candidates are incorrect. In this paper, we propose a novel inference scaling strategy, CoT-based Synthesizer, which leverages CoT reasoning to synthesize superior answers by analyzing complementary information from multiple candidate responses, even when all candidate responses are flawed. To enable a lightweight and cost-effective implementation, we introduce an automated data generation pipeline that creates diverse training data. This allows smaller LLMs trained on this data to improve the inference accuracy of larger models, including API-based LLMs. Experimental results across four benchmark datasets with seven policy models demonstrate that our method significantly enhances performance, with gains of 11.8% for Llama3-8B and 10.3% for GPT-4o on the MATH dataset. The corresponding training data and code are publicly available on https://github.com/RUCKBReasoning/CoT-based-Synthesizer.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑