反复提问:探究大语言模型对重复问题的鲁棒性
Asking Again and Again: Exploring LLM Robustness to Repeated Questions
AI总结:
本文研究在提示中重复问题对 LLM 性能的影响,在三个阅读理解数据集上评估五个模型,发现重复最多可提升 6% 准确率,但结果不具统计显著性,表明仅靠重复不能显著改善输出质量。
摘要:
本研究探讨在提示中重复问题是否会影响大语言模型(LLMs)的性能。我们假设在单个提示中重申问题可能会增强模型对查询关键要素的关注。我们在三个阅读理解数据集上评估了五个近期的大语言模型——包括 GPT-4o-mini、DeepSeek-V3 和较小的开源模型——并在不同的提示设置下,改变问题重复级别(每个提示重复 1、3 或 5 次)。我们的结果表明,问题重复可以将模型准确率提高最多 6%。然而,在所有模型、设置和数据集中,我们未发现该结果具有统计显著性。这些发现为提示设计和 LLM 行为提供了见解,表明仅靠重复并不会显著影响输出质量。
英文摘要:
This study investigates whether repeating questions within prompts influences the performance of large language models (LLMs). We hypothesize that reiterating a question within a single prompt might enhance the model's focus on key elements of the query. We evaluate five recent LLMs -- including GPT-4o-mini, DeepSeek-V3, and smaller open-source models -- on three reading comprehension datasets under different prompt settings, varying question repetition levels (1, 3, or 5 times per prompt). Our results demonstrate that question repetition can increase models' accuracy by up to $6\%$. However, across all models, settings, and datasets, we do not find the result statistically significant. These findings provide insights into prompt design and LLM behavior, suggesting that repetition alone does not significantly impact output quality.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。