句法框架脆弱性:LLM伦理决策稳健性审计
Syntactic Framing Fragility: An Audit of Robustness in LLM Ethical Decisions
AI总结:
本文提出Syntactic Framing Fragility框架,用于量化LLM在逻辑等价句法变换下的决策一致性,发现开源模型在高风险决策中表现更脆弱,且否定性语法是主要失效模式。
摘要:
大型语言模型表现出系统性的否定敏感性,但缺乏在部署规模上衡量此脆弱性的操作框架,尤其是在高风险决策中。我们引入句法框架脆弱性(SFF),一种用于量化决策一致性在逻辑等价句法变换下的框架。SFF通过逻辑极性归一化隔离句法影响,能够在正负框架间直接比较并控制极性反转,提供句法变化指数(SVI)作为稳健性度量,适用于CI/CD集成。审计23个模型在14个高风险场景(39,975个决策)中,建立了现象的地面真实效应大小,发现开源模型的脆弱性比商业模型高2.2倍。否定性语法是主要失效模式,某些模型在被问及是否不行动时仍以80-97%的频率支持行动。这些模式与先前工作中记录的否定抑制失败一致,链式推理在某些情况下减少脆弱性,但并非所有情况。我们提供了场景分层的风险概况,并提供符合欧盟AI法案和NIST RMF要求的操作检查清单。代码、数据和场景将在发表时发布。
英文摘要:
Large language models exhibit systematic negation sensitivity, yet no operational framework exists to measure this vulnerability at deployment scale, especially in high-stakes decisions. We introduce Syntactic Framing Fragility (SFF), a framework for quantifying decision consistency under logically equivalent syntactic transformations. SFF isolates syntactic effects via Logical Polarity Normalization, enabling direct comparison across positive and negative framings while controlling for polarity inversion, and provides the Syntactic Variation Index (SVI) as a robustness metric suitable for CI/CD integration. Auditing 23 models across 14 high-stakes scenarios (39,975 decisions), we establish ground-truth effect sizes for a phenomenon previously characterized only qualitatively and find that open-source models exhibit $2.2x higher fragility than commercial counterparts. Negation-bearing syntax is the dominant failure mode, with some models endorsing actions at 80-97% rates even when asked whether agents not act. These patterns are consistent with negation suppression failure documented in prior work, with chain-of-thought reasoning reducing fragility in some but not all cases. We provide scenario-stratified risk profiles and offer an operational checklist compatible with EU AI Act and NIST RMF requirements. Code, data, and scenarios will be released upon publication.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。