少样本困境：过度提示大型语言模型
The Few-shot Dilemma: Over-prompting Large Language Models
AI总结：
本文提出一个提示框架，使用随机采样、语义嵌入和TF-IDF三种少样本选择方法，在多个LLM上实验发现过多领域特定示例会降低性能，并通过TF-IDF与分层采样结合找到最优示例数量，在软件需求分类上超越现有方法1%。
摘要：
过度提示是一种现象，即提示中过多的示例导致大型语言模型（LLMs）性能下降，挑战了关于上下文少样本学习的传统观点。为了研究这种少样本困境，我们概述了一个提示框架，该框架利用三种标准的少样本选择方法——随机采样、语义嵌入和TF-IDF向量——并在多个LLM上评估这些方法，包括GPT-4o、GPT-3.5-turbo、DeepSeek-V3、Gemma-3、LLaMA-3.1、LLaMA-3.2和Mistral。我们的实验结果表明，在提示中加入过多的领域特定示例可能会在某些LLM中反常地降低性能，这与先前认为更多相关少样本示例普遍有利于LLM的实证结论相矛盾。鉴于LLM辅助软件工程和需求分析的趋势，我们在两个真实世界的软件需求分类数据集上进行了实验。通过逐步增加TF-IDF选择和分层的少样本示例数量，我们为每个LLM确定了其最优数量。这种组合方法以更少的示例实现了更优的性能，避免了过度提示问题，从而在功能性和非功能性需求分类上超越了现有技术1%。
英文摘要：
Over-prompting, a phenomenon where excessive examples in prompts lead to diminished performance in Large Language Models (LLMs), challenges the conventional wisdom about in-context few-shot learning. To investigate this few-shot dilemma, we outline a prompting framework that leverages three standard few-shot selection methods - random sampling, semantic embedding, and TF-IDF vectors - and evaluate these methods across multiple LLMs, including GPT-4o, GPT-3.5-turbo, DeepSeek-V3, Gemma-3, LLaMA-3.1, LLaMA-3.2, and Mistral. Our experimental results reveal that incorporating excessive domain-specific examples into prompts can paradoxically degrade performance in certain LLMs, which contradicts the prior empirical conclusion that more relevant few-shot examples universally benefit LLMs. Given the trend of LLM-assisted software engineering and requirement analysis, we experiment with two real-world software requirement classification datasets. By gradually increasing the number of TF-IDF-selected and stratified few-shot examples, we identify their optimal quantity for each LLM. This combined approach achieves superior performance with fewer examples, avoiding the over-prompting problem, thus surpassing the state-of-the-art by 1% in classifying functional and non-functional requirements.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
