arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2407.11511cs.AIcs.CLcs.LG
收藏

大语言模型的多步推理:综述

Multi-Step Reasoning with Large Language Models, a Survey

LIACS(莱顿大学信息科学研究中心);Leiden University(莱顿大学)

Aske Plaat, Annie Wong, Suzan Verberne, Joost Broekens, Niki van Stein, Thomas Back

AI总结:

本文综述了大语言模型的多步推理领域,提出了生成、评估和控制多步推理的分类法,指出该方法已超越数学应用题扩展至逻辑、博弈和机器人领域,并总结了强化学习微调等核心方法与未来研究议程。

摘要:

具有数十亿参数的大型语言模型(LLMs)展现出上下文学习能力,使其能够在未专门训练的任务上实现少样本学习。传统模型在语言任务上取得了突破性表现,但在基础推理基准上表现不佳。然而,一种新的上下文学习方法,即思维链(Chain-of-thought),在这些基准上展示了强大的多步推理能力。关于LLM推理能力的研究始于LLM能否解决小学数学应用题的问题,并在过去几年中扩展到了其他任务。本文综述了利用LLM进行多步推理的领域。我们提出了一个分类法,用于识别生成、评估和控制多步推理的不同方式。我们深入探讨了核心方法和开放性问题,并为不久的将来提出了研究议程。我们发现,多步推理方法已经超越了数学应用题,现在能够成功解决逻辑、组合博弈和机器人技术中的挑战,有时是通过首先生成代码然后由外部工具执行来实现的。多步方法中的许多研究使用强化学习进行微调、外部优化循环、上下文强化学习和自我反思。

英文摘要:

Large language models (LLMs) with billions of parameters exhibit in-context learning abilities, enabling few-shot learning on tasks that the model was not specifically trained for. Traditional models achieve breakthrough performance on language tasks, but do not perform well on basic reasoning benchmarks. However, a new in-context learning approach, Chain-of-thought, has demonstrated strong multi-step reasoning abilities on these benchmarks. The research on LLM reasoning abilities started with the question whether LLMs can solve grade school math word problems, and has expanded to other tasks in the past few years. This article reviews the field of multi-step reasoning with LLMs. We propose a taxonomy that identifies different ways to generate, evaluate, and control multi-step reasoning. We provide an in-depth coverage of core approaches and open problems, and we propose a research agenda for the near future. We find that multi-step reasoning approaches have progressed beyond math word problems, and can now successfully solve challenges in logic, combinatorial games, and robotics, sometimes by first generating code that is then executed by external tools. Many studies in multi-step methods use reinforcement learning for finetuning, external optimization loops, in-context reinforcement learning, and self-reflection.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑