arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2604.22348cs.LG
收藏

全国日本医疗索赔数据集的医疗基础模型:在模型扩展与任务特定计算效率之间平衡

A Nationwide Japanese Medical Claims Foundation Model: Balancing Model Scaling and Task-Specific Computational Efficiency

Graduate School of Medicine, Kyoto University(京都大学医学研究科);Medical Data Vision Co., Ltd.(医疗数据视觉株式会社);IQVIA Solutions Japan G.K.(IQVIA日本解决方案公司);DeSC Healthcare, Inc.(DeSC医疗公司)

Nanae Aratake, Taisei Tosaki, Yuji Okamoto, Eiichiro Uchino, Masaki Nakamura, Nobutomo Matsui, Akiko Hatakama, Yasushi Okuno

AI总结:

研究通过日本全国519家医院的索赔数据库,评估不同模型规模对结构化医疗数据下游任务性能的影响,发现任务特性决定最优模型规模,从而平衡预测性能与计算成本。

摘要:

利用纵向医疗数据进行临床风险预测支持个体化医疗。自监督基础模型已成为利用大规模未标记医疗记录的有希望的方法。在自然语言处理中,扩展定律表明更大模型能更可预测地降低预训练损失,支持基础模型范式。然而,对于结构化医疗数据,其特征是词汇量有限且观察稀疏,是否增加模型大小能一致提高下游预测尚不明确,因为大多数研究只评估单一模型规模。在本研究中,我们评估了结构化医疗基础模型中模型规模与下游任务性能的关系。使用来自日本全国519家医院索赔数据库的随机样本(230万患者,32家医院),我们对疾病发病率和药物预测进行了预训练,模型规模为2.2M-101M参数。下游性能在任务依赖的阈值下饱和:疾病预测受益于更大模型(32M-101M),而药物预测在11M时饱和,减少预训练时间178小时。在所有任务中,表现最佳的模型在精确率-召回率曲线下面积上始终优于Light Gradient Boosting Machine基线。这些发现表明,不同于单调递减的预训练损失,最佳模型规模取决于任务特性。这种任务依赖的饱和为在结构化医疗基础模型中平衡预测性能和计算成本提供了实用指导。

英文摘要:

Clinical risk prediction using longitudinal medical data supports individualized care. Self-supervised foundation models have emerged as a promising approach for leveraging large-scale unlabeled healthcare records. In natural language processing, scaling laws suggest that larger models achieve predictably lower pretraining losses, supporting the foundation model paradigm. However, for structured medical data, characterized by a limited vocabulary and sparse observations, whether increasing model size consistently improves downstream predictions is unclear, as most studies evaluate only a single model scale. In this study, we evaluated the relationship between model scale and downstream task performance for structured medical foundation models. Using a random sample (2.3 million patients, 32 hospitals) from a nationwide 519-hospital Japanese claims database, we pretrained encoder-only Transformers at five scales (2.2M-101M parameters) for disease incidence and medication prediction. Downstream performance saturated at task-dependent thresholds: disease prediction benefited from larger models (32M-101M), whereas medication prediction saturated at 11M, reducing pretraining time by 178 h. Across all tasks, the best-performing model consistently outperformed a Light Gradient Boosting Machine baseline in the area under the precision-recall curve. These findings indicate that, unlike the monotonically decreasing pretraining loss, the optimal model size varied depending on task characteristics. This task-dependent saturation provides practical guidance for balancing predictive performance and computational cost in structured medical foundation models.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑