arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2607.08054cs.LGcs.AI
收藏

谁来分析分析器?使用结构化元安全分析过程的自验证大语言模型风险分析

Who Analyses the Analyser? Self-Validating LLM Hazard Analysis with Constitutional Meta-STPA

Samuel Tetteh, Udip Shrestha, Joshua R. Waite, Cody Fleming

AI总结:

研究针对大语言模型辅助安全分析工具自身缺乏分析的问题,提出结构化元安全分析过程,通过运行元安全分析过程推导治理章程,包含21条工具原则和8条元安全原则,还报告了自推导等四个发现。

摘要:

大语言模型(LLMs)在诸如系统理论过程分析(STPA)等严格流程中,越来越被信任来起草安全分析工件,如损失、风险、不安全控制行动(UCAs)和安全约束。然而,快速增长的文献存在一个盲点:每个系统都被分析了,除了进行分析的LLM辅助工具本身,它是一个与安全相关的系统,可能会产生幻觉标准、发出无法验证的约束,并且从提示到工件没有审计跟踪。我们认真对待该领域跳过的问题——谁来分析分析器?并通过将STPA应用于工具本身来回答。我们提出了结构化元安全分析过程(Constitutional Meta-STPA),这是一个围绕闭环构建的LLM辅助STPA工具:该工具运行一类人工智能辅助安全工具的元安全分析过程,并从产生的损失→风险→UCA→约束链中推导而非断言其治理章程,产生了一份包含21条工具原则和8条元安全原则的已发布章程,每条原则都与一个代码执行点相关联。我们将测量对象形式化为一个在原则集P(|P| = 29)上的章程边际覆盖算子,并给出一个将覆盖与模型和扫描器分离的健全性引理,并报告了四个发现。(i)自推导:一个前沿集成(claude-opus-4.8 + claude-sonnet-4)从工具自身设计中恢复了18/21条规范原则和所有8/8条治理原则,而一个较弱的组合恢复了12/21条和3/8条,因此元层受模型限制,而非章程限制,并且相同的8/8条原则从第二个独立编写的工具中再次出现。

英文摘要:

Large language models (LLMs) are increasingly trusted to draft the artifacts of safety analysis such as, losses, hazards, Unsafe Control Actions (UCAs), and safety constraints, inside rigorous processes such as Systems-Theoretic Process Analysis (STPA). Yet a blind spot runs through this fast-growing literature: every system gets analysed except the LLM-assisted tool doing the analysing, which is itself a safety-relevant system that can hallucinate standards, emit unverifiable constraints, and leave no audit trail from prompt to artifact. We take seriously the question the field has skipped -- {who analyses the analyser?} and answer it by turning STPA on the tool itself. We present \{Constitutional Meta-STPA}, an LLM-assisted STPA tool built around a closed loop: the tool runs a {meta-STPA} of the class of AI-assisted safety tools and {derives} rather than asserts, its governance constitution from the resulting loss$\to$hazard$\to$UCA$\to$constraint chain, yielding a published constitution of $21$ Tool Principles and $8$ Meta-Safety Principles, each bound to a code enforcement point. We formalise the measured object as a constitution-marginal coverage operator over a principle set $P$ ($|P|{=}29$) with a soundness lemma that isolates coverage from model and scanner, and report four findings. {(i)~Self-derivation:} a frontier ensemble ({claude-opus-4.8}${+}${claude-sonnet-4}) recovers $18/21$ canonical and all $8/8$ governance principles from the tool's own design, while a weaker pair recovers $12/21$ and $3/8$, so the meta layer is model-limited, not constitution-limited, and the same $8/8$ re-emerge from a second, independently authored tool.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑