arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2310.01441cs.CLcs.AI
收藏

UPAR:一种受康德启发、用于增强大语言模型能力的提示框架

UPAR: A Kantian-Inspired Prompting Framework for Enhancing Large Language Model Capabilities

University of California, Santa Barbara(加州大学圣巴巴拉分校)

Hejia Geng, Boxun Xu, Peng Li

AI总结:

本文提出受康德先验哲学启发的 UPAR 四阶段提示框架,通过理解、规划、行动和反思增强 LLM 推理,并在 GSM8K、因果判断及 SCIBENCH 上验证其性能提升。

摘要:

大语言模型(LLM)已经展现出令人瞩目的推理能力,许多研究工作致力于通过提示来增强这种能力。尽管已有这些努力,但一个统一的认识论基础仍然明显缺失。受康德先验哲学启发,我们提出 UPAR 提示框架,旨在在 LLM 中模拟人类认知的结构。UPAR 框架被划分为四个阶段:“理解(Understand)”“规划(Plan)”“行动(Act)”和“反思(Reflect)”,从而能够从复杂语境中提取结构化信息、预先规划解决方案、按计划执行并进行自我反思。该结构显著提升了 LLM 推理的可解释性和准确性,产生人类可理解且可检查的推理轨迹。此外,我们的工作为现有提示技术提供了认识论基础,使这些方法有可能实现系统化整合。借助 GPT-4,在 GSM8K 的一个具有挑战性的子集上,我们的方法将准确率从 COT 基线的 22.92% 提升到 58.33%;在因果判断任务中,将准确率从 67.91% 提升到 75.40%。在不使用少样本示例或外部工具的情况下,UPAR 在 SCIBENCH 上显著优于现有提示方法;SCIBENCH 是一个具有挑战性的数据集,包含大学层级的数学、化学和物理科学问题。

英文摘要:

Large Language Models (LLMs) have demonstrated impressive inferential capabilities, with numerous research endeavors devoted to enhancing this capacity through prompting. Despite these efforts, a unified epistemological foundation is still conspicuously absent. Drawing inspiration from Kant's a priori philosophy, we propose the UPAR prompting framework, designed to emulate the structure of human cognition within LLMs. The UPAR framework is delineated into four phases: "Understand", "Plan", "Act", and "Reflect", enabling the extraction of structured information from complex contexts, prior planning of solutions, execution according to plan, and self-reflection. This structure significantly augments the explainability and accuracy of LLM inference, producing a human-understandable and inspectable inferential trajectory. Furthermore, our work offers an epistemological foundation for existing prompting techniques, allowing for a possible systematic integration of these methods. With GPT-4, our approach elevates the accuracy from COT baseline of 22.92% to 58.33% in a challenging subset of GSM8K, and from 67.91% to 75.40% in the causal judgment task. Without using few-shot examples or external tools, UPAR significantly outperforms existing prompting methods on SCIBENCH, a challenging dataset containing collegiate-level mathematics, chemistry, and physics scientific problems.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑