arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2402.18252cs.CLcs.AI
收藏

基于心智模型的大语言模型通用提示方法探索

Towards Generalist Prompting for Large Language Models by Mental Models

University of Science and Technology of China(中国科学技术大学)

Haoxiang Guan, Jiyan He, Shuxin Zheng, En-Hong Chen, Weiming Zhang, Nenghai Yu

AI总结:

本文提出通用提示概念及 MeMo 方法,通过将多种提示核心提炼为心智模型并由 LLM 自主选择,在零样本多类推理任务上达到或接近最优。

摘要:

大语言模型(LLM)已在许多任务上展现出令人瞩目的性能。然而,为了获得最佳性能,仍需采用专门设计的提示方法。这些方法要么依赖于需要一定领域知识的、特定于任务的少样本示例,要么设计得较为简单,却只在少数几类任务上表现良好。在本研究中,我们尝试引入通用提示(generalist prompting)的概念,其设计原则是在广泛的任务范围内取得最优或接近最优的性能,同时无需为特定问题人工选择和定制提示。此外,我们提出了 MeMo(Mental Models,心智模型),这是一种设计简单却能有效满足通用提示标准的创新提示方法。MeMo 将各种提示方法的核心提炼为独立的心智模型,并允许 LLM 针对问题自主选择最合适的心智模型,在零样本设置下,于 STEM、逻辑推理和常识推理等多种任务上达到或接近当前最优结果。我们希望本文提出的洞见能够推动对 LLM 通用提示方法的进一步探索。

英文摘要:

Large language models (LLMs) have demonstrated impressive performance on many tasks. However, to achieve optimal performance, specially designed prompting methods are still needed. These methods either rely on task-specific few-shot examples that require a certain level of domain knowledge, or are designed to be simple but only perform well on a few types of tasks. In this work, we attempt to introduce the concept of generalist prompting, which operates on the design principle of achieving optimal or near-optimal performance on a wide range of tasks while eliminating the need for manual selection and customization of prompts tailored to specific problems. Furthermore, we propose MeMo (Mental Models), an innovative prompting method that is simple-designed yet effectively fulfills the criteria of generalist prompting. MeMo distills the cores of various prompting methods into individual mental models and allows LLMs to autonomously select the most suitable mental models for the problem, achieving or being near to the state-of-the-art results on diverse tasks such as STEM, logical reasoning, and commonsense reasoning in zero-shot settings. We hope that the insights presented herein will stimulate further exploration of generalist prompting methods for LLMs.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑