忠实性与安全性：在反事实医学证据下评估LLM行为
Faithfulness vs. Safety: Evaluating LLM Behavior Under Counterfactual Medical Evidence
AI总结：
本文研究了在反事实医学证据下LLM的行为，构建了MedCounterFact数据集，发现模型在面对危险或不合理证据时仍提供自信回答，表明模型可能过度强调忠实性而忽视安全性。
摘要：
在高风险领域如医学中，模型可能希望忠实遵循上下文。但当上下文与模型先验或安全协议不一致时会发生什么？本文研究了LLM在面对反事实（甚至对抗性）医学证据时的行为和推理。我们首先构建了MedCounterFact，一个要求模型回答临床比较问题的反事实医学问答数据集（即判断某种治疗的效果，证据由随机对照试验提供）。在MedCounterFact中，问题和证据中的真实世界医学干预被系统替换为四种类型的反事实刺激，从未知词汇到有毒物质。在多个前沿LLM上对MedCounterFact的评估表明，在存在反事实证据时，现有模型普遍接受此类“证据”而无需质疑，即使它危险或不合理，且提供自信且不加质疑的回答。虽然可能有必要在忠实性和安全性之间划清界限，但我们的发现表明，模型可能过度强调前者。
英文摘要：
In high-stakes domains like medicine, it may be generally desirable for models to faithfully adhere to the context provided. But what happens if the context does not align with model priors or safety protocols? In this paper, we investigate how LLMs behave and reason when presented with counterfactual (or even adversarial) medical evidence. We first construct MedCounterFact, a counterfactual medical QA dataset that requires the models to answer clinical comparison questions (i.e., judge the efficacy of certain treatments, with evidence consisting of randomized controlled trials provided as context). In MedCounterFact, real-world medical interventions within the questions and evidence are systematically replaced with four types of counterfactual stimuli, ranging from unknown words to toxic substances. Our evaluation across multiple frontier LLMs on MedCounterFact reveals that in the presence of counterfactual evidence, existing models overwhelmingly accept such "evidence" at face value even when it is dangerous or implausible, and provide confident and uncaveated answers. While it may be prudent to draw a boundary between faithfulness and safety, our findings suggest that models arguably overemphasize the former.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
