数学问题求解中思维链的设计
Design of Chain-of-Thought in Math Problem Solving
AI总结：
本文系统比较自然语言与不同风格、编程语言的程序 CoT，在 GSM8K、MATHQA 和 SVAMP 上验证程序 CoT 更优，并给出设计指导。
摘要：
思维链（Chain-of-Thought，CoT）在数学问题求解的推理中发挥着关键作用。我们对设计 CoT 的方法进行了全面考察，将传统自然语言 CoT 与多种程序 CoT 进行比较，包括自描述程序、注释描述程序和无描述程序。此外，我们研究了编程语言对程序 CoT 的影响，比较了 Python 和 Wolfram Language。通过在 GSM8K、MATHQA 和 SVAMP 上的大量实验，我们发现程序 CoT 在数学问题求解中通常具有更优的有效性。值得注意的是，表现最佳的 30B 参数组合以显著优势击败了 GPT-3.5-turbo。结果表明，自描述程序提供了更高的多样性，因此通常能够实现更高的性能。我们还发现，对于程序 CoT 而言，Python 是比 Wolfram 更好的语言选择。实验结果为未来的 CoT 设计提供了有价值的指导，即在进一步发展中同时考虑编程语言和编码风格。我们的数据集和代码已公开发布。
英文摘要：
Chain-of-Thought (CoT) plays a crucial role in reasoning for math problem solving. We conduct a comprehensive examination of methods for designing CoT, comparing conventional natural language CoT with various program CoTs, including the self-describing program, the comment-describing program, and the non-describing program. Furthermore, we investigate the impact of programming language on program CoTs, comparing Python and Wolfram Language. Through extensive experiments on GSM8K, MATHQA, and SVAMP, we find that program CoTs often have superior effectiveness in math problem solving. Notably, the best performing combination with 30B parameters beats GPT-3.5-turbo by a significant margin. The results show that self-describing program offers greater diversity and thus can generally achieve higher performance. We also find that Python is a better choice of language than Wolfram for program CoTs. The experimental results provide a valuable guideline for future CoT designs that take into account both programming language and coding style for further advancements. Our datasets and code are publicly available.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
