arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2211.13892cs.CL
收藏

面向有效上下文学习的互补性解释

Complementary Explanations for Effective In-Context Learning

The University of Texas at Austin（德克萨斯大学奥斯汀分校）；Meta AI

Xi Ye, Srinivasan Iyer, Asli Celikyilmaz, Ves Stoyanov, Greg Durrett, Ramakanth Pasunuru

备注：
ACL Findings 2023 Camera-Ready
阅读 PDF
arXiv 页面 ↗HTML 原文 ↗

AI总结：

本研究探究上下文学习中解释的作用机制，发现计算轨迹与提示语言均影响解释效果，且多样化互补解释可提升性能，进而提出基于最大边际相关性的示例选择方法，在多模型多任务上有效提升上下文学习表现。

摘要：

大语言模型（LLM）已展现出从提示中的解释进行学习的卓越能力，但人们对这些解释的具体作用机制或其生效原因的理解仍十分有限。本研究旨在更深入地揭示解释用于上下文学习的作用机制。我们首先研究了两个不同因素对带解释提示的性能的影响：计算轨迹（解的分解方式）以及用于表达提示的自然语言。通过在三个受控任务上对解释进行扰动，我们表明这两个因素都对解释的有效性有贡献。我们进一步研究了如何构建用于解决给定测试查询的最高效解释集。研究发现，LLM 可以从解释集的互补性中获益：不同示例所展示的多样化推理技能能够带来更优的性能。因此，我们提出了一种基于最大边际相关性的示例选择方法，用于构建既具相关性又具互补性的示例集，该方法在多个 LLM 上的三个真实世界任务中成功提升了上下文学习性能。

英文摘要：

Large language models (LLMs) have exhibited remarkable capabilities in learning from explanations in prompts, but there has been limited understanding of exactly how these explanations function or why they are effective. This work aims to better understand the mechanisms by which explanations are used for in-context learning. We first study the impact of two different factors on the performance of prompts with explanations: the computation trace (the way the solution is decomposed) and the natural language used to express the prompt. By perturbing explanations on three controlled tasks, we show that both factors contribute to the effectiveness of explanations. We further study how to form maximally effective sets of explanations for solving a given test query. We find that LLMs can benefit from the complementarity of the explanation set: diverse reasoning skills shown by different exemplars can lead to better performance. Therefore, we propose a maximal marginal relevance-based exemplar selection approach for constructing exemplar sets that are both relevant as well as complementary, which successfully improves the in-context learning performance across three real-world tasks on multiple LLMs.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

↑