解码层面禁忌：大语言模型鲁棒性的诊断压力测试

中文摘要 AI 辅助

大语言模型评估通常聚焦于标称条件下的性能，营造出模型在狭窄、高度优化的生成通道中轻松运行的能力假象。然而在实际部署中，复杂的系统提示、安全护栏和结构约束持续迫使模型偏离标称路径，导致基准测试分数与部署性能之间产生偏差。为解决该问题，我们提出了Decoding-Level Taboo（解码层面禁忌），这是一种零提示诊断压力测试，在运行时直接干预logit空间，迫使模型偏离标称路径。通过在单词边界处动态掩码主要候选token，Taboo迫使模型进行迂回表达。在多个开源权重模型系列上对Taboo进行评估后发现，偏离路径的鲁棒性同时受参数规模和后训练指令对齐的影响，且鲁棒性通常随模型规模和对齐程度的提升而提高。除本文呈现的结果外，Taboo还为生成多样化合成数据集、压力测试运行时安全护栏以及在实际部署前审计模型可靠性提供了一种新的基础原语。

英文摘要

Large language model evaluations typically focus on performance under nominal conditions, creating an illusion of capability where models comfortably walk a narrow, highly optimized generation corridor. In real-world deployments, however, complex system prompts, safety guardrails, and structural constraints continuously force models off this nominal path, driving a divergence between benchmark scores and deployment performance. To address this issue, we introduce Decoding-Level Taboo, a zero-prompt diagnostic stress test that intervenes directly in logit space at runtime, forcing models out of their nominal paths. By dynamically masking primary candidate tokens at word boundaries, Taboo forces machine circumlocution. Evaluating Taboo across several open-weight model families reveals that off-path robustness is heavily influenced by both parameter scale and post-training instruction alignment, with robustness generally improving with model size and alignment. Beyond the results presented in this paper, Taboo provides a novel primitive for generating diverse synthetic datasets, stress-testing runtime safety guardrails, and auditing model reliability prior to real-world deployment.

发表机构

• University of the Ryukyus（琉球大学）
• Technion（以色列理工学院）
• Universidad Polécnica de Madrid（马德里理工大学）

机构由 AI 辅助整理，请以论文原文为准。