arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2406.13846cs.CLcs.LG
收藏

文本序列化及其与表格机器学习传统范式的关系

Text Serialization and Their Relationship with the Conventional Paradigms of Tabular Machine Learning

University of California, Los Angeles(加利福尼亚大学洛杉矶分校);International Christian University(国际基督教大学)

Kyoka Ono, Simon A. Lee

AI总结:

本研究对比文本序列化结合语言模型的方法与表格机器学习传统范式,从数据表示和分类性能两方面展开实验,发现当前预训练模型尚无法替代传统表格机器学习方法。

摘要:

近期研究探索了语言模型(Language Models, LMs)如何用于表格机器学习任务中的特征表示与预测,这涉及文本序列化和监督微调(supervised fine-tuning, SFT)技术。尽管这些技术原理简单,但我们对语言模型在此场景下的适用性和可靠性的认知仍存在显著空白。本研究评估了新兴的语言模型技术与表格机器学习传统范式的对比情况,并验证采用这类先进技术实施相似方案的可行性。在数据层面,我们研究了序列化表格数据的多种数据表示与整理方法,探究其对预测性能的影响;在分类层面,我们检验了文本序列化结合语言模型是否能提升表格数据集(如类别不平衡、分布偏移、偏差和高维场景)上的性能,并评估该方法是否属于解决表格机器学习挑战的 state-of-the-art(SOTA,当前最优)方法。研究结果表明,当前的预训练模型不应取代传统方法。

英文摘要:

Recent research has explored how Language Models (LMs) can be used for feature representation and prediction in tabular machine learning tasks. This involves employing text serialization and supervised fine-tuning (SFT) techniques. Despite the simplicity of these techniques, significant gaps remain in our understanding of the applicability and reliability of LMs in this context. Our study assesses how emerging LM technologies compare with traditional paradigms in tabular machine learning and evaluates the feasibility of adopting similar approaches with these advanced technologies. At the data level, we investigate various methods of data representation and curation of serialized tabular data, exploring their impact on prediction performance. At the classification level, we examine whether text serialization combined with LMs enhances performance on tabular datasets (e.g. class imbalance, distribution shift, biases, and high dimensionality), and assess whether this method represents a state-of-the-art (SOTA) approach for addressing tabular machine learning challenges. Our findings reveal current pre-trained models should not replace conventional approaches.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑