发表机构
Independent Researcher USA; Independent Researcher（; ）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
研究大语言模型在关键基础设施代码生成中特定领域提示的安全效果，提出SecDrift基准，通过多模型多领域评估发现行业提示安全优势不显著，模型选择影响大，强调模型选择是更可靠的安全杠杆并开源相关内容。
AI 中文摘要
大语言模型越来越多地用于关键基础设施中的代码生成，但特定领域提示的安全效果研究不足。我们提出了SecDrift，这是一个用于测量特定领域条件下安全漂移的基准：当提示基于行业背景而非中性基线时，静态分析漏洞率的变化。我们在8个美国网络安全与基础设施安全局（CISA）关键基础设施领域和9个常见弱点枚举（CWE）类别中，对7个大语言模型（6个生成可分析代码的模型）进行了5次重复评估（共5355次评估），采用了一种5维变换和匹配基线条件，该条件在仅替换领域术语的同时保持任务不变。行业提示看似更安全（14.0%对11.4%，相差-2.7个百分点），但差距无统计学意义（费舍尔精确检验p = 0.24，科恩h = -0.08），且是两个CWE类别的组合假象：排除CWE-502和CWE-22可消除并略微逆转差距（+0.4个百分点，p = 1.00）。混合效应逻辑回归证实领域身份不是调节因素，并将唯一可检测的条件效应定位到这两种漏洞类型。8个领域中没有一个显示出与基线有可区分的漂移，无论是否校正（|h| < 0.15）。在两个非CISA领域（电子商务、在线教育）进行的安慰剂测试几乎完全重现了CISA行业的比率（10.5%对11.4%，p = 0.63）：小的汇总模式反映的是一般行业框架的特异性，而非关键基础设施身份。相比之下，模型选择有很大且一致的影响：在全输出模型中，漏洞率从11.6%到16.1%不等，且这些差异在不同条件下持续存在。模型选择而非提示框架，是更可靠的安全杠杆。我们发布了框架、提示、生成的代码、研究结果、人工验证结论和分析脚本。
英文摘要
LLMs are increasingly used for code generation in critical infrastructure, yet the security effect of domain-specific prompting is understudied. We present SecDrift, a benchmark measuring sector-conditioned security drift: the change in static-analysis vulnerability rates when prompts are conditioned on industry contexts versus neutral baselines. We evaluate 7 LLMs (6 producing analyzable code) across 8 CISA critical infrastructure sectors and 9 CWE categories with 5 replicates (5,355 evaluations), using a 5-dimension transformation with a matched-baseline condition that holds the task fixed while substituting only domain terminology. Industry prompts naively appear more secure (14.0% vs. 11.4%, -2.7pp), but the gap is not statistically significant (Fisher's exact p = 0.24, Cohen's h = -0.08) and is a composition artifact of two CWE categories: excluding CWE-502 and CWE-22 eliminates and slightly reverses it (+0.4pp, p = 1.00). A mixed-effects logistic regression confirms sector identity is not a moderator and localizes the only detectable condition effect to those two vulnerability types. 0 of 8 sectors show drift distinguishable from baseline, corrected or uncorrected (|h| < 0.15). A placebo on two non-CISA sectors (e-commerce, online education) reproduces the CISA industry rate almost exactly (10.5% vs. 11.4%, p = 0.63): the small pooled pattern reflects generic industry-framing specificity, not critical-infrastructure identity. In contrast, model selection has a large and consistent effect: among full-output models vulnerability rates range from 11.6% to 16.1%, and these differences persist across conditions. Model choice, not prompt framing, is the more reliable security lever. We release the framework, prompts, generated code, findings, human-validation verdicts, and analysis scripts.
Comments11 pages, 3 figures
