面向大语言模型中自适应推理的思维回滚方法
Toward Adaptive Reasoning in Large Language Models with Thought Rollback
AI总结：
本文提出思维回滚（TR）框架，使大语言模型能对推理步骤进行错误分析并回滚修正，通过试错提示自适应构建更可靠的推理路径，在数学和多任务推理上以更低交互成本取得最先进求解率。
摘要：
大语言模型（LLMs）已被广泛用于通过逐步推理来解决各种任务。然而，中间推理步骤（即思维）的结构是僵化且单向的，例如链式、树形或无环有向图。因此，这种不灵活且仅向前推进的推理可能无法应对具有挑战性的任务，并且当LLM频繁给出错误回答（即“幻觉”）时会失败。本文提出了一种新的推理框架，称为思维回滚（Thought Rollback，TR），使LLM能够在“幻觉”情况下自适应地构建思维结构，同时保持面向问题求解的有效推理。TR的核心机制是回滚思维，它允许LLM对思维进行错误分析，从而回滚到任何先前出错的思维进行修正。随后，通过在提示中包含这种试错过程来引导LLM，每次回滚都会产生一条更可靠的推理路径。因此，从一个无需人工标注的简单提示开始，带有TR的LLM能够自适应地逐步探索思维以获得正确解答。在数学问题和多任务推理上的综合实验表明，TR在问题求解率和交互成本方面达到了最先进的性能。例如，带有TR的GPT-4在MATH数据集上的求解率比当前最佳方法高出9%。
英文摘要：
Large language models (LLMs) have been routinely used to solve various tasks using step-by-step reasoning. However, the structure of intermediate reasoning steps, or thoughts, is rigid and unidirectional, such as chains, trees, or acyclic-directed graphs. Consequently, the resulting inflexible and forward-only reasoning may not address challenging tasks and fail when the LLM frequently gives false responses, i.e., ``hallucinations''. This paper proposes a new reasoning framework, called Thought Rollback (TR), allowing LLMs to adaptively build thought structure while maintaining effective reasoning toward problem-solving under ``hallucinations''. The core mechanism of TR is rolling back thoughts, which allows LLMs to perform error analysis on thoughts, and thus roll back to any previously mistaken thought for revision. Subsequently, by including such trial-and-error in the prompt to guide the LLM, each rollback leads to one more reliable reasoning path. Therefore, starting with a simple prompt without human annotations, LLM with TR adaptively and gradually explores thoughts for a correct solution. Comprehensive experiments on mathematical problems and multi-task reasoning demonstrate the state-of-the-art performance of TR in terms of problem-solving rate and interaction cost. For instance, the solving rate of GPT-4 with TR outperforms the current best by $9\%$ on the MATH dataset.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
