arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2501.06863cs.LGcs.AIcs.CL
收藏

通过微调大语言模型实现表格数据的迁移学习

Transfer Learning of Tabular Data by Finetuning Large Language Models

Tennessee State University(田纳西州立大学)

Shourav B. Rabbani, Ibna Kowsar, Manar D. Samad

AI总结:

针对表格数据深度学习缺乏有效迁移学习的问题,本文提出端到端微调LLM的迁移学习方法,在10个基准数据集上验证其在少特征表格数据分类上优于现有方法,且计算成本更低。

摘要:

尽管人工智能(AI)掀起了革命浪潮,但深度学习在表格数据领域尚未取得太多成功,原因在于特征空间异构、样本量有限且缺乏可行的迁移学习方法。由大语言模型(LLM)驱动的生成式AI新时代,为各类数据和领域带来了前所未有的学习机遇。本文研究了LLM应用程序编程接口(API)以及LLM迁移学习在表格数据分类任务中的有效性。LLM API通过输入文本提示(包含标记化数据与指令)返回响应,而迁移学习则是针对目标分类任务微调LLM。本文提出了一种端到端的LLM微调方法,在不存在可辅助迁移学习的大型预训练表格数据模型的情况下,在10个基准数据集上验证了跨数据迁移学习的效果。所提出的LLM微调方法在特征数少于10个(这是表格数据集的典型特征规模)的表格数据任务上,表现优于当前最优的机器学习与深度学习方法。该迁移学习方法的计算成本仅为其他深度学习或基于API的解决方案的一小部分,同时能保证具备竞争力甚至更优的分类性能。

英文摘要:

Despite the artificial intelligence (AI) revolution, deep learning has yet to achieve much success with tabular data due to heterogeneous feature space and limited sample sizes without viable transfer learning. The new era of generative AI, powered by large language models (LLM), brings unprecedented learning opportunities to diverse data and domains. This paper investigates the effectiveness of an LLM application programming interface (API) and transfer learning of LLM in tabular data classification. LLM APIs respond to input text prompts with tokenized data and instructions, whereas transfer learning finetunes an LLM for a target classification task. This paper proposes an end-to-end finetuning of LLM to demonstrate cross-data transfer learning on ten benchmark data sets when large pre-trained tabular data models do not exist to facilitate transfer learning. The proposed LLM finetuning method outperforms state-of-the-art machine and deep learning methods on tabular data with less than ten features - a standard feature size for tabular data sets. The transfer learning approach uses a fraction of the computational cost of other deep learning or API-based solutions while ensuring competitive or superior classification performance.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑