元认知提示提升大型语言模型的理解能力
Metacognitive Prompting Improves Understanding in Large Language Models
AI总结：
本研究提出受人类内省推理启发的元认知提示（MP）策略，通过结构化自我认知评估提升大型语言模型理解能力，在十个NLU数据集上验证其优于现有提示方法。
摘要：
在大型语言模型（LLMs）中，特定任务的性能不断提升，这在很大程度上受有效提示设计的影响。提示技术的最新进展增强了LLMs在逻辑密集型任务中的推理能力，然而这些模型在处理和解释复杂信息时所需的细致理解能力仍未被充分探索。在本研究中，我们引入了元认知提示（MP），这是一种受人类内省推理过程启发的策略。使用MP时，LLMs会经历一系列系统性的结构化自我认知评估，同时利用其庞大的内在知识和新见解。我们在四个主流LLMs：Llama2、PaLM2、GPT-3.5和GPT-4上进行了广泛实验，涵盖了来自GLUE、SuperGLUE、BLUE和LexGLUE基准的十个自然语言理解（NLU）数据集。此外，我们将该方法与思维链提示及其高级版本进行了比较。结果表明，GPT-4在所有任务中始终表现优异，而其他模型在与MP结合使用时在部分任务上取得了显著进步。此外，MP在通用和特定领域的NLU任务中均持续优于现有提示方法。本研究强调了增强LLMs理解能力的潜力，并突出了在NLU任务中模拟人类内省推理的益处。
英文摘要：
In Large Language Models (LLMs), there have been consistent advancements in task-specific performance, largely influenced by effective prompt design. Recent advancements in prompting have enhanced reasoning in logic-intensive tasks for LLMs, yet the nuanced understanding abilities of these models, crucial for processing and interpreting complex information, remain underexplored. In this study, we introduce Metacognitive Prompting (MP), a strategy inspired by human introspective reasoning processes. Using MP, LLMs undergo a systematic series of structured, self-aware evaluations, drawing on both their vast inherent knowledge and new insights. We conduct extensive experiments on four prevalent LLMs: Llama2, PaLM2, GPT-3.5, and GPT-4, across ten natural language understanding (NLU) datasets from GLUE, SuperGLUE, BLUE, and LexGLUE benchmarks. Additionally, we compare our method with chain-of-thought prompting and its advanced versions. The results show that GPT-4 consistently excels across all tasks, while other models have shown significant progress in some tasks when used in conjunction with MP. Furthermore, MP consistently outperforms existing prompting methods in both general and domain-specific NLU tasks. This study underscores the potential to amplify the understanding abilities of LLMs and highlights the benefits of mirroring human introspective reasoning in NLU tasks.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
