医学人工智能中的记忆偏差
Memorisation bias in medical AI
- Technical University of Munich (TUM)（慕尼黑工业大学）
- TUM University Hospital（慕尼黑工业大学附属医院）
- Munich Center for Machine Learning（慕尼黑机器学习中心）
- Imperial College London（伦敦帝国理工学院）
- Hasso Plattner Institute（哈索·普拉特纳研究院）
机构由 AI 辅助整理，请以论文原文为准。
AI总结：
本研究揭示医学AI模型因训练时见过患者历史数据，导致对其未来数据预测产生“记忆偏差”，影响诊断准确性，凸显隐私保护与模型部署间的矛盾。
AI中文摘要：
医学人工智能模型在改善患者预后方面具有巨大潜力，但已知它们会无意中记忆训练数据集中的个体记录。虽然这种记忆现象已被关联到针对性的隐私攻击，但其对临床部署的影响——即患者可能由在训练期间见过其历史数据的模型进行评估——仍知之甚少。在此，我们表明，如果模型在训练期间观察了同一患者的匿名历史数据，那么对该患者未见过的未来数据的预测可能会发生显著变化，我们将这一现象称为“记忆偏差”。我们证明，这种偏差存在于多种数据模态和模型架构中，并持续较长时间：在某些情况下，记忆偏差会持续存在于训练所用历史记录之后数十年获取的未来记录上。此外，在模拟的前瞻性部署中，记忆偏差对回归数据贡献者的诊断准确性具有不对称影响。当患者带着训练数据集中历史记录中不存在的新发疾病回归时，与未在其历史数据上训练的相同模型相比，诊断敏感性显著降低。相反，当其健康状况未改变时，敏感性和特异性均显著虚高。我们的发现揭示了医学人工智能中一种先前未被表征的风险，即当模型部署在为其训练数据做出贡献的患者身上时会出现这种风险。这暴露了当前模型开发实践的一个缺陷：旨在保护患者隐私的去标识化措施使得识别回归贡献者并将其排除在对其自身未来数据的AI辅助解读之外变得困难。因此，缓解记忆风险可能需要改变当前的模型训练和部署协议。
英文摘要：
Medical AI models hold immense potential to improve patient outcomes, but they are also known to unintentionally memorise individual records from their training datasets. While such memorisation has been linked to targeted privacy attacks, its consequences for clinical deployment, where patients may be assessed by a model that saw their historical data during training, remain poorly understood. Here we show that predictions on a patient's unseen future data can change significantly if a model observed that same patient's anonymised historical data during training, a phenomenon we term "memorisation bias". We demonstrate that this bias exists across diverse data modalities and model architectures, and over prolonged time spans: in some cases, memorisation bias persists on future records acquired decades after the historical records used for training. Moreover, in simulated prospective deployment, memorisation bias has asymmetric effects on the diagnostic accuracy of returning data contributors. When a patient returned with a de novo condition absent from their historical records in the training dataset, diagnostic sensitivity decreased significantly compared to an otherwise identical model not trained on their historical data. Conversely, when their health state was unchanged, both sensitivity and specificity were significantly inflated. Our findings reveal a previously uncharacterised risk in medical AI that arises when a model is deployed on patients who contributed to its training data. This exposes a shortcoming of current model development practice: the de-identification measures designed to protect patients' privacy make it difficult to identify returning contributors and exclude them from the AI-assisted interpretation of their own future data. Mitigating memorisation risks may thus require changes to current model training and deployment protocols.
