AI 中文总结
本研究发现，在医疗资源分配场景中，大语言模型的推理设置会导致其对相同患者信息产生不同的资源分配概率，凸显了谨慎将LLM系统纳入决策的重要性。
AI 中文摘要
大语言模型正被纳入几乎所有领域的敏感且重要的决策过程中。此前的研究关注模型围绕输入和场景框架的偏差，而模型也可能因部署过程中积累的上下文表现出意外且不可取的行为。本研究以医疗场景为例：给定简短临床背景，要求模型为两人分配资源分配概率，之后向模型展示相同场景，但新增一句包含对比患者信息的句子，模型的上下文是否包含其之前的响应分两种情况。在测试的四个模型中，有三个模型的配对上下文实验与独立推理实验存在不同的概率变化，当新增信息加入时，变化方向通常相反（分别倾向于B对象或A对象）。我们还开展了额外的配对上下文实验，以展示场景维度属性变化的影响。研究结果表明，在敏感医疗用例中，患者信息存在依赖上下文的影响；更广泛而言，本研究凸显了将基于大语言模型的系统谨慎纳入决策过程、开展上下文工程以及推进模型行为研究的重要性。
英文摘要
Large language models are being incorporated into sensitive and important decision-making processes across nearly all fields. While prior work studies model bias around inputs and scenario framing, models can also behave in unexpected and undesirable ways due to context accumulated over their deployment. In this work, we study a medical example in which a model is asked to assign resource-allocation probabilities to two people given brief clinical context, and then sees the same scenario with a single extra sentence containing contrasting patient information, either with or without its previous response in context. Across three of four tested models, the paired-context and independent-inference experiments have different probability shifts, often in opposite directions (in favor of Person B vs. in favor of Person A) when new information is provided. We include additional paired-context experiments to show the effect of varying attributes across scenario axes. Our findings show the context-dependent effect of patient information in a sensitive medical use case. More broadly, our work shows the importance of carefully incorporating LLM-based systems into decision-making processes, context engineering, and further model behavioral studies.
CommentsAccepted to the AI4GOOD Workshop at ICML 2026, Seoul, South Korea
