arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2603.07460cs.CRcs.AI
收藏

LLM-based系统在哪里失效?一个系统级安全框架用于风险评估与处理

Where Do LLM-based Systems Break? A System-Level Security Framework for Risk Assessment and Treatment

Northern Arizona University(北亚利桑那大学);Department of Software Science(软件科学系);Tallinn University of Technology(塔林技术大学)

Neha Nagaraja, Hayretdin Bahsi

AI总结:

本文提出了一种系统级安全框架,用于评估和处理LLM系统中的风险,通过结合系统建模与攻击-防御树,分析医疗场景中的多步骤攻击路径,并提供可比较的防御策略。

摘要:

大型语言模型(LLMs)正越来越多地集成到安全关键型工作流程中,但现有的安全分析仍然碎片化,往往将模型行为与更广泛系统背景隔离开来。本文介绍了一个以目标为导向的风险评估框架,用于LLM驱动的系统,结合系统建模与攻击-防御树(ADTrees)和基于常见漏洞评分系统(CVSS)的可利用性评分,以支持结构化、可比较的分析。我们通过一个医疗案例研究演示了该框架,建模多步骤攻击路径,针对医疗程序中的干预、电子健康记录(EHR)数据泄露和服务可用性中断。我们的分析表明,威胁涵盖(i)传统网络、(ii)对抗性机器学习、(iii)操纵提示或上下文的对话攻击,往往集中于少数主导路径和共享系统瓶颈,使针对性防御能够显著降低路径可利用性。通过系统比较防御组合,我们将这些风险与已建立的漏洞管理实践对齐,并提供一个适用于其他LLM关键系统的领域无关工作流程。

英文摘要:

Large Language Models (LLMs) are increasingly integrated into safety-critical workflows, yet existing security analyses remain fragmented and often isolate model behavior from the broader system context. This work introduces a goal-driven risk assessment framework for LLM-powered systems that combines system modeling with Attack-Defense Trees (ADTrees) and Common Vulnerability Scoring System (CVSS)-based exploitability scoring to support structured, comparable analysis. We demonstrate the framework through a healthcare case study, modeling multi-step attack paths targeting intervention in medical procedures, leakage of electronic health record (EHR) data, and disruption of service availability. Our analysis indicates that threats spanning (i) conventional cyber, (ii) adversarial ML, and (iii) conversational attacks that manipulate prompts or context often consolidate into a small number of dominant paths and shared system choke points, enabling targeted defenses to yield meaningful reductions in path exploitability. By systematically comparing defense portfolios, we align these risks with established vulnerability management practices and provide a domain-agnostic workflow applicable to other LLM-enabled critical systems.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑