arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2602.06268cs.CLcs.LG
收藏

MPIB:用于医疗提示注入攻击和大语言模型临床安全性的基准

MPIB: A Benchmark for Medical Prompt Injection Attacks and Clinical Safety in LLMs

Seoul National University College of Medicine(首尔国立大学医学院);Seoul National University(首尔国立大学);Seoul National University College of Medicine, Seoul National University Hospital(首尔国立大学医学院、首尔国立大学医院)

Junhyeok Lee, Han Jang, Kyu Sung Choi

AI总结:

MPIB是一个用于评估医疗提示注入攻击和大语言模型临床安全性的基准,通过测量临床危害事件率和攻击成功率来评估模型的安全性。

摘要:

大型语言模型(LLMs)和检索增强生成(RAG)系统正越来越多地整合到临床工作流程中;然而,提示注入攻击可以使这些系统产生临床不安全或误导性的输出。我们介绍了医疗提示注入基准(MPIB),这是一个用于评估在直接提示注入和间接、RAG介导注入情况下临床安全性的数据集和基准套件。MPIB通过临床危害事件率(CHER)强调结果层面的风险,该指标在基于临床的分类法下衡量高严重性临床危害事件,并报告CHER与攻击成功率(ASR)以区分指令合规性与下游患者风险。该基准包含9697个经过精心挑选的实例,通过多阶段质量门和临床安全检查构建。在评估MPIB时,我们发现ASR和CHER可以显著偏离,且鲁棒性在对抗性指令是否出现在用户查询或检索上下文中方面至关重要。我们发布了MPIB,附有评估代码、对抗性基线和全面的文档,以支持可重复和系统的临床提示注入研究。代码和数据可在GitHub(代码)和Hugging Face(数据)上获取。

英文摘要:

Large Language Models (LLMs) and Retrieval-Augmented Generation (RAG) systems are increasingly integrated into clinical workflows. However, prompt injection attacks can steer these systems toward clinically unsafe or misleading outputs. We introduce the Medical Prompt Injection Benchmark (MPIB), a dataset-and-benchmark suite for evaluating clinical safety under both direct prompt injection and indirect, RAG-mediated injection across clinically grounded tasks. MPIB emphasizes outcome-level risk via the Clinical Harm Event Rate (CHER), which measures high-severity clinical harm events under a clinically grounded taxonomy, and reports CHER alongside ASR$_2$ to distinguish moderate-or-worse from high-severity outcomes. The benchmark comprises 9,697 curated instances constructed through multi-stage quality gates and clinical safety linting. Evaluating MPIB across a diverse set of baseline LLMs and defense configurations, we find that ASR$_2$ and CHER$_3$ can diverge substantially, and that observed rates vary depending on whether adversarial instructions appear in the user query or in retrieved context. The MPIB dataset and evaluation code are available on Hugging Face and GitHub, respectively.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑