arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2603.23987cs.LG
收藏

能否利用LLMs为临床时间序列数据生成可移植的表示?

Can we generate portable representations for clinical time series data using LLMs?

University of Toronto, Canada(加拿大多伦多大学);Sunnybrook Health Sciences Centre, Canada(加拿大阳光医疗科学中心);Vector Institute, Canada(加拿大向量研究所)

Zongliang Ji, Yifei Sun, Andre Amaral, Anna Goldenberg, Rahul G. Krishnan

AI总结:

本文研究了利用LLMs生成可移植患者表示的方法,通过将不规则ICU时间序列转换为自然语言摘要,再嵌入固定长度向量,实现跨医院的预测任务,展示了其在多个临床任务中的竞争力和可移植性。

摘要:

在临床机器学习中,部署缓慢且易碎:在一家医院表现良好的模型在另一家医院往往因分布偏移而退化。本文研究了一个简单问题:大型语言模型(LLMs)能否生成可移植的患者表示,即允许在一家医院构建的下游预测器在其他地方使用时几乎不需要重新训练和微调。为此,我们通过冻结的LLM将不规则ICU时间序列映射到简洁的自然语言摘要,然后用冻结的文本嵌入模型嵌入每个摘要,以获得固定长度的向量,可作为多种下游预测器的输入。在三个队列(MIMIC-IV、HIRID、PPICU)上,针对多个临床相关的预测和分类任务,我们发现我们的方法简单、易于使用,并且在分布内与网格插补、自监督表示学习和时间序列基础模型竞争,同时在转移到新医院时表现出较小的相对性能下降。我们研究了性能在提示设计上的变化,发现结构化提示对于减少预测模型的方差至关重要,而不影响平均准确率。我们发现使用这些可移植的表示可以提高少样本学习,并且与基线相比,年龄和性别的可恢复性没有增加,表明隐私风险很小。我们的工作指出LLMs作为工具的潜力,可以减少工程开销,从而实现生产级预测模型的可扩展部署。

英文摘要:

Deploying clinical ML is slow and brittle: models that work at one hospital often degrade under distribution shifts at the next. In this work, we study a simple question -- can large language models (LLMs) create portable patient embeddings i.e. representations of patients enable a downstream predictor built on one hospital to be used elsewhere with minimal-to-no retraining and fine-tuning. To do so, we map from irregular ICU time series onto concise natural language summaries using a frozen LLM, then embed each summary with a frozen text embedding model to obtain a fixed length vector capable of serving as input to a variety of downstream predictors. Across three cohorts (MIMIC-IV, HIRID, PPICU), on multiple clinically grounded forecasting and classification tasks, we find that our approach is simple, easy to use and competitive with in-distribution with grid imputation, self-supervised representation learning, and time series foundation models, while exhibiting smaller relative performance drops when transferring to new hospitals. We study the variation in performance across prompt design, with structured prompts being crucial to reducing the variance of the predictive models without altering mean accuracy. We find that using these portable representations improves few-shot learning and does not increase demographic recoverability of age or sex relative to baselines, suggesting little additional privacy risk. Our work points to the potential that LLMs hold as tools to enable the scalable deployment of production grade predictive models by reducing the engineering overhead.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑