arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2501.01264cs.CLcs.AIcs.LG
收藏

ProgCo:程序助力大型语言模型的自我纠正

ProgCo: Program Helps Self-Correction of Large Language Models

Taobao & Tmall Group of Alibaba(阿里巴巴淘宝天猫集团)

Xiaoshuai Song, Yanan Wu, Weixun Wang, Jiaheng Liu, Wenbo Su, Bo Zheng

AI总结:

ProgCo 通过程序驱动验证与优化,解决 LLM 在复杂推理中自我纠正失败的问题,提升性能并开源代码。

摘要:

自我纠正旨在使大型语言模型(LLMs)无需外部反馈即可自我验证并优化其初始响应。然而,LLMs 往往无法有效自我验证并生成正确反馈,进而误导优化过程,导致自我纠正失败,尤其在复杂推理任务中。本文提出程序驱动的自我纠正(ProgCo)。首先,程序驱动验证(ProgVe)通过自生成、自执行的验证伪程序实现复杂验证逻辑和广泛验证。随后,程序驱动优化(ProgRe)接收来自 ProgVe 的反馈,对响应和验证程序进行双重反思与优化,以减轻复杂推理任务中错误反馈的误导。在三个指令遵循和数学基准上的实验表明,ProgCo 实现了有效的自我纠正,并与真实程序工具结合时能进一步提升性能。我们已在 https://github.com/songxiaoshuai/progco 发布代码。

英文摘要:

Self-Correction aims to enable large language models (LLMs) to self-verify and self-refine their initial responses without external feedback. However, LLMs often fail to effectively self-verify and generate correct feedback, further misleading refinement and leading to the failure of self-correction, especially in complex reasoning tasks. In this paper, we propose Program-driven Self-Correction (ProgCo). First, program-driven verification (ProgVe) achieves complex verification logic and extensive validation through self-generated, self-executing verification pseudo-programs. Then, program-driven refinement (ProgRe) receives feedback from ProgVe, conducts dual reflection and refinement on both responses and verification programs to mitigate misleading of incorrect feedback in complex reasoning tasks. Experiments on three instruction-following and mathematical benchmarks indicate that ProgCo achieves effective self-correction, and can be further enhance performance when combined with real program tools. We release our code at https://github.com/songxiaoshuai/progco.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑