对医疗预测中黑箱大语言模型的代理建模解释
Surrogate modeling for interpreting black-box LLMs in medical predictions
AI总结：
本文提出一种代理建模框架，通过输入输出对近似复杂系统，揭示LLM编码知识的范围，揭示医疗预测中潜在的不准确和偏见。
摘要：
大型语言模型（LLMs）在大量数据集上训练，其参数中编码了广泛的真实世界知识，但其黑箱性质掩盖了编码机制和范围。代理建模通过简化模型近似复杂系统，可为黑箱模型的可解释性提供路径。我们提出了一种代理建模框架，定量解释LLM编码的知识。对于特定的假设，该框架通过广泛的模拟场景提示，利用可观测元素（输入输出对）近似潜在的LLM知识空间。通过概念验证实验在医疗预测中，我们展示了该框架在揭示LLM对每个输入变量与输出关系的感知范围的有效性。特别是，鉴于LLM可能延续其训练数据中嵌入的不准确性和社会偏见，我们的实验使用该框架定量揭示了与已确立的医学知识相矛盾的关联以及科学上已被推翻的种族偏见在LLM编码知识中的持续存在。通过披露这些问题，该框架可以作为红灯指标，支持这些模型的安全可靠应用。
英文摘要：
Large language models (LLMs), trained on vast datasets, encode extensive real-world knowledge within their parameters, yet their black-box nature obscures the mechanisms and extent of this encoding. Surrogate modeling, which uses simplified models to approximate complex systems, can offer a path toward better interpretability of black-box models. We propose a surrogate modeling framework that quantitatively explains LLM-encoded knowledge. For a specific hypothesis derived from domain knowledge, this framework approximates the latent LLM knowledge space using observable elements (input-output pairs) through extensive prompting across a comprehensive range of simulated scenarios. Through proof-of-concept experiments in medical predictions, we demonstrate our framework's effectiveness in revealing the extent to which LLMs "perceive" each input variable in relation to the output. Particularly, given concerns that LLMs may perpetuate inaccuracies and societal biases embedded in their training data, our experiments using this framework quantitatively revealed both associations that contradict established medical knowledge and the persistence of scientifically refuted racial assumptions within LLM-encoded knowledge. By disclosing these issues, our framework can act as a red-flag indicator to support the safe and reliable application of these models.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
