arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2402.15729cs.AIcs.CLcs.PL
收藏

人类如何编写代码?大型模型也同样如此

How Do Humans Write Code? Large Models Do It the Same Way Too

East China Normal University(华东师范大学);Central Conservatory of Music(中央音乐学院);INF Technology(INF科技)

Long Li, Xuzheng He, Haozhe Wang, Linlin Wang, Liang He

AI总结:

针对思维程序在数学推理中引入更多错误的问题,提出人类思维语言(HTL),整合思维链与思维程序,通过新生成范式、焦点注意力和强化学习,在多个数据集上显著提升推理准确率并展现强迁移性。

摘要:

思维程序(PoT)通过利用外部工具调用来规避计算错误,取代了基于自然语言的思维链(CoT),成为大型语言模型(LLM)数学推理任务中最流行的方法。然而,我们对GPT-4和Llama系列的评估显示,与CoT相比,使用PoT会引入更多的推理错误,例如不正确的公式或有缺陷的逻辑。为了解决这个问题,我们提出了人类思维语言(HTL),它利用一系列策略来帮助整合PoT和CoT,包括:(1)一种新的生成范式,使用完整的CoT推理来控制代码生成。(2)焦点注意力,在PoT期间将模型注意力引导到CoT推理上,以生成更具逻辑性的代码。(3)强化学习,利用CoT和PoT响应的准确性作为奖励,以防止LLM在解决困难数学问题时出现重复的推理步骤。我们的方法在8个数学计算数据集上,在Llama-Base模型上平均提高了6.5%,在Mistral-Base模型上平均提高了4.3%。通过控制模型的信息流,它在五个域外数据集上也显示出显著的有效性,表现出很强的可迁移性。此外,HTL在非数学的自然语言推理任务中显示出最显著的改进,有助于构建统一的推理任务框架。

英文摘要:

Program-of-Thought (PoT) replaces natural language-based Chain-of-Thought (CoT) as the most popular method in Large Language Models (LLMs) mathematical reasoning tasks by utilizing external tool calls to circumvent computational errors. However, our evaluation of the GPT-4 and Llama series reveals that using PoT introduces more reasoning errors, such as incorrect formulas or flawed logic, compared to CoT. To address this issue, we propose Human-Think Language (HTL), which leverages a suite of strategies that help integrate PoT and CoT, encompassing: (1) a new generation paradigm that uses full CoT reasoning to control code generation. (2) Focus Attention, that directs model attention to the CoT reasoning during PoT to generate more logical code. (3) reinforcement learning that utilizes the accuracy of both CoT and PoT responses as rewards to prevent repetitive reasoning steps in LLMs when solving difficult math problems. Our method achieves an average improvement of 6.5% on the Llama-Base model and 4.3% on the Mistral-Base model across 8 mathematical calculation datasets. It also shows significant effectiveness on five out-of-domain datasets by controlling the model's information flow, exhibiting strong transferability. Additionally, HTL shows the most significant improvement in non-mathematical natural language inference task, contributing to a unified reasoning task framework

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑