发表机构
School of Computer Science and Technology, Tongji University（同济大学计算机科学与技术学院）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
本文提出基于交互作用的IPS指标，分析50个开源LLMs的提示敏感性，发现监督微调等四个因素可通过降低低阶交互作用的敏感性减少提示波动。
AI 中文摘要
大语言模型（LLMs）的卓越能力常受其不稳定性影响，即便是提示中细微且语义无关的变化也会引发性能的剧烈波动，这一现象被称为提示敏感性。过往研究通常通过比较提示变化时LLM的最终输出来评估提示敏感性，但这类粗粒度指标无法解释提示敏感性的内在原因。本文引入交互作用作为细粒度工具分析LLMs的提示敏感性，具体将LLM的输出分数分解为一组交互作用，每个交互作用代表涉及一组输入变量的非线性关系。研究发现，即便LLM的输出保持不变，提示的细微变化也会触发交互作用的严重不稳定性。为此，本文提出基于交互作用的提示敏感性（IPS）指标，通过量化提示发生细微变化时交互作用的变化来衡量敏感性。将IPS指标应用于50个开源LLMs后，研究揭示了可降低LLMs提示敏感性的四个因素：监督微调、更大的模型规模、密集架构和少样本学习。更关键的是，研究发现这四个因素降低提示敏感性的共同机制：它们均倾向于降低低阶交互作用（即涉及少量输入变量的交互作用）的提示敏感性。
英文摘要
The remarkable capabilities of large language models (LLMs) are often undermined by their instability. Even subtle and semantically irrelevant changes in prompts can cause dramatic fluctuations in performance, a phenomenon known as prompt sensitivity. Previous studies typically evaluate prompt sensitivity by comparing the LLM's final outputs when prompts change. However, such coarse-grained metrics fail to explain the internal reasons for prompt sensitivity. In this paper, we introduce interactions as a fine-grained tool to analyze prompt sensitivity of LLMs. Specifically, we decompose the output score of the LLM into a set of interactions. Each interaction represents a nonlinear relationship involving a set of input variables. We discover that subtle changes to prompts can trigger severe instability in interactions, even when the outputs of the LLM remain the same. To this end, we propose an Interaction-based Prompt Sensitivity (IPS) metric by quantifying changes in interactions when we introduce subtle changes to prompts. We apply the IPS metric to 50 open-source LLMs and uncover four factors that reduce the prompt sensitivity of LLMs, including supervised fine-tuning, increased model scales, dense architectures, and few-shot learning. More crucially, we discover a common mechanism by which these four factors reduce prompt sensitivity: all four factors tend to reduce the prompt sensitivity of low-order interactions (i.e., interactions involving few input variables).
CommentsAccepted at the 43rd International Conference on Machine Learning (ICML 2026). 46 pages, 48 figures
