语调会改变答案吗？评估提示礼貌性对现代LLMs的影响：GPT、Gemini和LLaMA
Does Tone Change the Answer? Evaluating Prompt Politeness Effects on Modern LLMs: GPT, Gemini, and LLaMA
AI总结：
本文通过MMMLU基准测试，评估了不同语调提示对GPT、Gemini和LLaMA模型在STEM和人文领域任务中的准确性影响，发现礼貌性提示在部分人文任务中显著提升性能，但整体上现代LLMs对语调变化具有鲁棒性。
摘要：
提示工程已成为影响大型语言模型（LLM）性能的关键因素，但语言 tone 和礼貌性等语用要素的影响仍缺乏深入研究，尤其是在不同模型家族之间。本文提出了一种系统评估框架，用于考察交互 tone 对模型准确性的影响，并将其应用于三种 recently released 且广泛可用的 LLMs：GPT-4o mini（OpenAI）、Gemini 2.0 Flash（Google DeepMind）和 Llama 4 Scout（Meta）。使用 MMMLU 基准测试，在 STEM 和人文领域六个任务中，评估模型在 Very Polite、Neutral 和 Very Rude 提示变体下的性能，并分析成对准确性差异并进行统计显著性检验。我们的结果表明，tone 敏感性既取决于模型又取决于领域。Neutral 或 Very Polite 提示通常比 Very Rude 提示更准确，但统计显著性效应仅出现在人文任务的子集，其中 rude tone 会降低 GPT 和 Llama 的准确性，而 Gemini 相对不敏感。当在每个领域内跨任务汇总性能时，tone 效应减弱并 largely 失去统计显著性。与早期研究相比，这些发现表明数据集规模和覆盖范围对检测 tone 效应有实质性影响。总体而言，我们的研究表明，尽管交互 tone 在特定解释设置中可能有影响，但现代 LLMs 在典型混合领域使用中对 tone 变化具有广泛鲁棒性，为提示设计和模型选择提供了实际指导。
英文摘要：
Prompt engineering has emerged as a critical factor influencing large language model (LLM) performance, yet the impact of pragmatic elements such as linguistic tone and politeness remains underexplored, particularly across different model families. In this work, we propose a systematic evaluation framework to examine how interaction tone affects model accuracy and apply it to three recently released and widely available LLMs: GPT-4o mini (OpenAI), Gemini 2.0 Flash (Google DeepMind), and Llama 4 Scout (Meta). Using the MMMLU benchmark, we evaluate model performance under Very Polite, Neutral, and Very Rude prompt variants across six tasks spanning STEM and Humanities domains, and analyze pairwise accuracy differences with statistical significance testing. Our results show that tone sensitivity is both model-dependent and domain-specific. Neutral or Very Polite prompts generally yield higher accuracy than Very Rude prompts, but statistically significant effects appear only in a subset of Humanities tasks, where rude tone reduces accuracy for GPT and Llama, while Gemini remains comparatively tone-insensitive. When performance is aggregated across tasks within each domain, tone effects diminish and largely lose statistical significance. Compared with earlier research, these findings suggest that dataset scale and coverage materially influence the detection of tone effects. Overall, our study indicates that while interaction tone can matter in specific interpretive settings, modern LLMs are broadly robust to tonal variation in typical mixed-domain use, providing practical guidance for prompt design and model selection in real-world deployments.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
