重新审视大型语言模型在文本分类中的提示敏感性：提示不足的作用
Revisiting Prompt Sensitivity in Large Language Models for Text Classification: The Role of Prompt Underspecification
AI总结：
本文研究了大型语言模型在文本分类中提示敏感性的来源，发现提示不足是导致性能波动的主要原因，并通过实验分析揭示了其对模型内部表示的影响。
摘要：
大型语言模型（LLMs）被广泛用作零样本和少样本分类器，其中任务行为主要通过提示进行控制。越来越多的研究发现，LLMs对提示变化非常敏感，微小的变化会导致性能显著变化。然而，在许多情况下，对敏感性的研究是通过提供最小任务指示和弱约束模型输出空间的不充分提示进行的。在本文中，我们认为观察到的提示敏感性在很大程度上可以归因于提示不足。我们系统地研究并比较了不充分提示和提供具体指示的提示的敏感性。利用性能分析、logit分析和线性探测，我们发现不充分提示表现出更高的性能方差和相关标记的更低logit值，而指示提示则较少受到此类问题的影响。然而，线性探测分析表明，提示不足的影响对LLM内部表示只有轻微影响，而是出现在最终层。总体而言，我们的发现强调了在研究和缓解提示敏感性时需要更加严谨。
英文摘要：
Large language models (LLMs) are widely used as zero-shot and few-shot classifiers, where task behaviour is largely controlled through prompting. A growing number of works have observed that LLMs are sensitive to prompt variations, with small changes leading to large changes in performance. However, in many cases, the investigation of sensitivity is performed using underspecified prompts that provide minimal task instructions and weakly constrain the model's output space. In this work, we argue that a significant portion of the observed prompt sensitivity can be attributed to prompt underspecification. We systematically study and compare the sensitivity of underspecified prompts and prompts that provide specific instructions. Utilising performance analysis, logit analysis, and linear probing, we find that underspecified prompts exhibit higher performance variance and lower logit values for relevant tokens, while instruction-prompts suffer less from such problems. However, linear probing analysis suggests that the effects of prompt underspecification have only a marginal impact on the internal LLM representations, instead emerging in the final layers. Overall, our findings highlight the need for more rigour when investigating and mitigating prompt sensitivity.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
