arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2311.08154cs.CLcs.AI
收藏

再问一次!自我一致性在(几乎)所有场景下提升语言模型的推理能力

Just Ask One More Time! Self-Agreement Improves Reasoning of Language Models in (Almost) All Scenarios

Kuaishou Technology(快手科技);School of Computer Science and Engineering, Northeastern University(东北大学计算机科学与工程学院)

Lei Lin, Jiayi Fu, Pengli Liu, Qingyang Li, Yan Gong, Junchen Wan, Fuzheng Zhang, Zhongyuan Wang, Di Zhang, Kun Gai

AI总结:

针对思维链贪心解码的局限,提出Self-Agreement方法,通过采样多样化推理路径并再次提示模型选择最一致答案,在六个基准上显著提升推理性能并具备强泛化能力。

摘要:

尽管思维链(chain-of-thought, CoT)提示结合语言模型在复杂推理任务上取得了令人鼓舞的结果,但CoT提示中使用的朴素贪心解码通常会导致重复性和局部最优问题。为了解决这一缺点,集成优化方法试图获取多条推理路径以进行最终答案的集成。然而,当前的集成优化方法要么简单地采用基于规则的后期处理,如\textit{自一致性}(self-consistency),要么基于若干任务相关的人工标注训练一个额外模型来从多条推理路径中选择最佳答案,但这些方法在输入问题类型未知或推理路径答案格式未知的现实场景中无法泛化。为了避免这些局限性,我们提出了\textbf{Self-Agreement}(自我一致性),这是一种可泛化的集成优化方法,适用于几乎所有场景,在这些场景中输入问题类型和推理路径的答案格式可能是已知或未知的。Self-Agreement首先从语言模型的解码器中采样以生成一组\textit{多样化}的推理路径,随后\textit{再次}提示语言模型,通过从采样的推理路径中选择最\textit{一致}的答案来确定最优答案。Self-Agreement在六个公开推理基准上同时取得了显著的表现和优越的泛化能力。

英文摘要:

Although chain-of-thought (CoT) prompting combined with language models has achieved encouraging results on complex reasoning tasks, the naive greedy decoding used in CoT prompting usually causes the repetitiveness and local optimality. To address this shortcoming, ensemble-optimization tries to obtain multiple reasoning paths to get the final answer assembly. However, current ensemble-optimization methods either simply employ rule-based post-processing such as \textit{self-consistency}, or train an additional model based on several task-related human annotations to select the best one among multiple reasoning paths, yet fail to generalize to realistic settings where the type of input questions is unknown or the answer format of reasoning paths is unknown. To avoid their limitations, we propose \textbf{Self-Agreement}, a generalizable ensemble-optimization method applying in almost all scenarios where the type of input questions and the answer format of reasoning paths may be known or unknown. Self-agreement firstly samples from language model's decoder to generate a \textit{diverse} set of reasoning paths, and subsequently prompts the language model \textit{one more time} to determine the optimal answer by selecting the most \textit{agreed} answer among the sampled reasoning paths. Self-agreement simultaneously achieves remarkable performance on six public reasoning benchmarks and superior generalization capabilities.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑