arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2310.10047cs.CL
收藏

改进大型语言模型微调以解决数学问题

Improving Large Language Model Fine-tuning for Solving Math Problems

Yale University(耶鲁大学);Google DeepMind(谷歌DeepMind)

Yixin Liu, Avi Singh, C. Daniel Freeman, John D. Co-Reyes, Peter J. Liu

AI总结:

针对LLM解数学题性能不足,提出三种微调策略(解答微调、解答簇重排序、多任务顺序微调),在MATH数据集上使PaLM 2-L准确率达58.8%,较基线提升11.2%。

摘要:

尽管大型语言模型(LLM)在许多自然语言任务中取得了成功,但解决数学问题仍然是其面临的重大挑战。LLM在解决数学问题时,一次通过(pass-at-one)与多次通过(pass-at-N)的性能之间存在巨大差距,这表明LLM可能接近找到正确答案,这促使我们探索微调方法以释放LLM的性能。利用具有挑战性的MATH数据集,我们研究了三种微调策略:(1)解答微调,即微调模型以针对给定数学问题生成详细解答;(2)解答簇重排序,即将LLM微调为解答验证器/评估器,从生成的候选解答簇中选择最佳解答;(3)多任务顺序微调,即高效地整合解答生成与评估任务以增强LLM性能。通过这些方法,我们对一系列PaLM 2模型进行了全面的实证研究,发现:(1)用于微调的逐步解答的质量和风格会对模型性能产生显著影响;(2)解答重排序和多数投票在单独使用时都能有效提升模型性能,但两者结合使用可带来更大的性能提升;(3)顺序分离解答生成与评估任务的多任务微调相比解答微调基线能提供更好的性能。在这些见解的指导下,我们设计了一种微调方案,在MATH数据集上使用微调后的PaLM 2-L模型达到了约58.8%的准确率,相比预训练PaLM 2-L模型采用多数投票的少样本性能提升了11.2%的准确率。

英文摘要:

Despite their success in many natural language tasks, solving math problems remains a significant challenge for large language models (LLMs). A large gap exists between LLMs' pass-at-one and pass-at-N performance in solving math problems, suggesting LLMs might be close to finding correct solutions, motivating our exploration of fine-tuning methods to unlock LLMs' performance. Using the challenging MATH dataset, we investigate three fine-tuning strategies: (1) solution fine-tuning, where we fine-tune to generate a detailed solution for a given math problem; (2) solution-cluster re-ranking, where the LLM is fine-tuned as a solution verifier/evaluator to choose among generated candidate solution clusters; (3) multi-task sequential fine-tuning, which integrates both solution generation and evaluation tasks together efficiently to enhance the LLM performance. With these methods, we present a thorough empirical study on a series of PaLM 2 models and find: (1) The quality and style of the step-by-step solutions used for fine-tuning can make a significant impact on the model performance; (2) While solution re-ranking and majority voting are both effective for improving the model performance when used separately, they can also be used together for an even greater performance boost; (3) Multi-task fine-tuning that sequentially separates the solution generation and evaluation tasks can offer improved performance compared with the solution fine-tuning baseline. Guided by these insights, we design a fine-tuning recipe that yields approximately 58.8% accuracy on the MATH dataset with fine-tuned PaLM 2-L models, an 11.2% accuracy improvement over the few-shot performance of pre-trained PaLM 2-L model with majority voting.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑