现有预条件器能否提升生物医学表格基础学习？TabPFN优化的实证研究

中文摘要 AI 辅助

表格基础模型近期在结构化生物医学数据分析中展现出强大潜力。其中，TabPFN已成为低数据量表格分类任务的有效方法。然而，优化和预条件策略对生物医学微调的影响在很大程度上仍未得到探索。在本工作中，我们对五种基于AdamW的预条件策略在59个生物医学数据集上微调TabPFN v2.5进行了全面的实证研究，这些数据集涵盖阿尔茨海默病、乳腺癌、精神分裂症、显著记忆担忧（SMC）、KEEL生物医学数据集以及UCI生物医学基准。评估考虑了预测性能、计算效率和统计显著性分析。实验结果表明，原始AdamW优化器始终取得最佳的整体性能和统计排名，而现有的曲率感知预条件器未能在多样化的生物医学学习场景中提供可靠的改进。这些发现表明，通用预条件方法可能无法充分捕捉生物医学表格学习的优化特性，从而激励开发专门针对医疗保健导向的表格基础模型的生物医学感知预条件器。

英文摘要

Tabular foundation models have recently shown strong potential for structured biomedical data analysis. Among them, TabPFN has emerged as an effective approach for low-data tabular classification tasks. However, the impact of optimization and preconditioning strategies on biomedical fine-tuning remains largely unexplored. In this work, we present a comprehensive empirical investigation of five AdamW-based preconditioning strategies for fine-tuning TabPFN v2.5 on 59 biomedical datasets spanning Alzheimer's disease, breast cancer, schizophrenia, significant memory concern (SMC), KEEL biomedical datasets, and UCI biomedical benchmarks. The evaluation considers predictive performance, computational efficiency, and statistical significance analysis. Experimental results demonstrate that the original AdamW optimizer consistently achieves the best overall performance and statistical ranking, while existing curvature-aware preconditioners fail to provide reliable improvements across diverse biomedical learning scenarios. The findings suggest that generic preconditioning approaches may not adequately capture the optimization characteristics of biomedical tabular learning, motivating the development of biomedical-aware preconditioners specifically tailored for healthcare-oriented tabular foundation models.

发表机构

• Indian Institute of Technology Indore（印度理工学院印多尔分校）
• University of Florence（佛罗伦萨大学）

机构由 AI 辅助整理，请以论文原文为准。