arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2401.05618cs.CLcs.AI
收藏

简洁思维链对大型语言模型问题解决的益处

The Benefits of a Concise Chain of Thought on Problem-Solving in Large Language Models

Johns Hopkins University(约翰斯·霍普金斯大学)

Matthew Renze, Erhan Guven

AI总结:

提出CCoT提示方法,并在GPT-3.5、GPT-4的MCQA基准上验证其可显著缩短回答、降低每token成本且总体保持性能。

摘要:

本文介绍了简洁思维链(Concise Chain-of-Thought,CCoT)提示方法。我们比较了标准CoT与CCoT提示,以考察简洁性如何影响回答长度和正确答案准确率。我们使用GPT-3.5和GPT-4,在一个多项选择问答(MCQA)基准上进行评估。CCoT使GPT-3.5和GPT-4的平均回答长度均减少48.70%,同时对问题解决性能的影响可以忽略。然而,在数学问题上,使用CCoT的GPT-3.5产生27.69%的性能损失。总体而言,CCoT带来22.67%的平均每token成本降低。所有代码、数据和补充材料均可在GitHub获取:https://github.com/matthewrenze/jhu-concise-cot

英文摘要:

In this paper, we introduce Concise Chain-of-Thought (CCoT) prompting. We compared standard CoT and CCoT prompts to see how conciseness impacts response length and correct-answer accuracy. We evaluated this using GPT-3.5 and GPT-4 with a multiple-choice question-and-answer (MCQA) benchmark. CCoT reduced average response length by 48.70% for both GPT-3.5 and GPT-4 while having a negligible impact on problem-solving performance. However, on math problems, GPT-3.5 with CCoT incurs a performance penalty of 27.69%. Overall, CCoT leads to an average per-token cost reduction of 22.67%. All code, data, and supplemental materials are available on GitHub at https://github.com/matthewrenze/jhu-concise-cot

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑