BenchOverflow: 通过纯文本提示测量大语言模型中的溢出现象
BenchOverflow: Measuring Overflow in Large Language Models via Plain-Text Prompts
AI总结：
BenchOverflow通过纯文本提示策略评估大语言模型的溢出现象，揭示长度控制对可靠性、成本和可持续性的影响，提供标准化比较框架以优化部署和防御措施。
摘要：
我们研究了大型语言模型（LLMs）中的一种故障模式，即纯文本提示会引发过度输出，这种现象我们称为溢出。与劫持或提示注入不同，溢出出现在普通的交互设置下，可能导致服务成本、延迟和跨用户性能下降，尤其是在多个请求中扩展时。除了可用性之外，其影响是经济和环境的：不必要的标记会增加每次请求的成本和能耗，随着规模扩大，会产生显著的操作支出和碳足迹。此外，溢出代表了在共享环境中计算放大和服务降级的实用向量。我们介绍了BenchOverflow，这是一个模型无关的基准，包含九种纯文本提示策略，这些策略在没有对抗性后缀或政策绕过的情况下放大输出体积。使用标准化的协议，固定预算为5000个新标记，我们评估了九个开源和闭源模型，并观察到长度分布出现明显的右移和重尾现象。容量饱和率（CSR@1k/3k/5k）和经验累积分布函数（ECDFs）量化尾部风险；在同一提示内的方差和跨模型相关性显示，溢出在广泛可重现但不同家族和攻击向量中存在异质性。一种轻量级的缓解措施——固定简洁性提示——能减轻右尾并降低所有策略在大多数模型中的CSR。我们的发现将长度控制定位为可测量的可靠性、成本和可持续性问题，而不是一种风格上的小瑕疵。通过使不同模型的长度控制鲁棒性进行标准化比较，BenchOverflow为选择最小化资源浪费和运营支出的部署提供了实际基础，并为评估能够遏制计算放大而不损害任务性能的防御措施提供了评估依据。
英文摘要：
We investigate a failure mode of large language models (LLMs) in which plain-text prompts elicit excessive outputs, a phenomenon we term Overflow. Unlike jailbreaks or prompt injection, Overflow arises under ordinary interaction settings and can lead to elevated serving cost, latency, and cross-user performance degradation, particularly when scaled across many requests. Beyond usability, the stakes are economic and environmental: unnecessary tokens increase per-request cost and energy consumption, compounding into substantial operational spend and carbon footprint at scale. Moreover, Overflow represents a practical vector for compute amplification and service degradation in shared environments. We introduce BenchOverflow, a model-agnostic benchmark of nine plain-text prompting strategies that amplify output volume without adversarial suffixes or policy circumvention. Using a standardized protocol with a fixed budget of 5000 new tokens, we evaluate nine open- and closed-source models and observe pronounced rightward shifts and heavy tails in length distributions. Cap-saturation rates (CSR@1k/3k/5k) and empirical cumulative distribution functions (ECDFs) quantify tail risk; within-prompt variance and cross-model correlations show that Overflow is broadly reproducible yet heterogeneous across families and attack vectors. A lightweight mitigation-a fixed conciseness reminder-attenuates right tails and lowers CSR for all strategies across the majority of models. Our findings position length control as a measurable reliability, cost, and sustainability concern rather than a stylistic quirk. By enabling standardized comparison of length-control robustness across models, BenchOverflow provides a practical basis for selecting deployments that minimize resource waste and operating expense, and for evaluating defenses that curb compute amplification without eroding task performance.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
