Recent research has explored how Language Models (LMs) can be used for feature representation and prediction in tabular machine learning tasks. This involves employing text serialization and supervised fine-tuning (SFT) techniques. Despite the simplicity of these techniques, significant gaps remain in our understanding of the applicability and reliability of LMs in this context. Our study assesses how emerging LM technologies compare with traditional paradigms in tabular machine learning and evaluates the feasibility of adopting similar approaches with these advanced technologies. At the data level, we investigate various methods of data representation and curation of serialized tabular data, exploring their impact on prediction performance. At the classification level, we examine whether text serialization combined with LMs enhances performance on tabular datasets (e.g. class imbalance, distribution shift, biases, and high dimensionality), and assess whether this method represents a state-of-the-art (SOTA) approach for addressing tabular machine learning challenges. Our findings reveal current pre-trained models should not replace conventional approaches.
Accept or Deny? Evaluating LLM Fairness and Performance in Loan Approval across Table-to-Text Serialization Approaches
Saarland University(萨尔兰大学);CISPA Helmholtz Center for Information Security(CISPA海德堡中心 for 信息安全);Max Planck Institute for Software Systems(马克斯·普朗克软件系统研究所)
本文评估三种地区贷款审批数据在不同表到文本序列化方式下的 LLM 零样本与 ICL 表现,发现格式显著影响 F1 与公平性。
Improving Robustness of Tabular Retrieval via Representational Stability
Department of Computer Science Rensselaer Polytechnic Institute(罗切斯特理工学院计算机科学系);Department of Computer Science Arizona State University(亚利桑那州立大学计算机科学系);ScaleAI
TableLoRA: Low-rank Adaptation on Table Structure Understanding for Large Language Models
State Key Laboratory of Human-Machine Hybrid Augmented Intelligence, Institute of Artificial Intelligence and Robotics, Xi’an Jiaotong University(人机混合增强智能国家重点实验室,人工智能与机器人研究院,西安交通大学);Peking University(北京大学);Microsoft(微软公司)
Evaluating LLM Abilities to Understand Tabular Electronic Health Records: A Comprehensive Study of Patient Data Extraction and Retrieval
Université Paul Sabatier(保罗·萨巴蒂埃大学);IRIT(信息论与电信研究所);Centre Hospitalier Universitaire de Toulouse(图卢兹大学医院中心);INSERM(法国国家健康与医学研究院);Université de Toulouse(图卢兹大学);Faculté de Médecine(医学院)
TAP4LLM: Table Provider on Sampling, Augmenting, and Packing Semi-structured Data for Large Language Model Reasoning
National University of Singapore(新加坡国立大学);University of Illinois Urbana-Champaign(伊利诺伊大学厄巴纳-香槟分校);Microsoft(微软);Xi’an Jiaotong University(西安交通大学);Ant Research(蚂蚁研究院)
Extract Information from Hybrid Long Documents Leveraging LLMs: A Framework and Dataset
Microsoft(微软公司);School of Software & Microelectronics, Peking University(北京大学软件与微电子学院);Institute of Software, Chinese Academy of Sciences(中国科学院软件研究所);Ant Group(蚂蚁集团)
Tabular LLMs for Interpretable Few-Shot Alzheimer's Disease Prediction with Multimodal Biomedical Data
University of Pennsylvania(宾夕法尼亚大学);Stony Brook University(石溪大学);University of North Carolina at Chapel Hill(北卡罗来纳大学教堂山分校);Medical University of South Carolina(南卡罗来纳医科大学);Cedars-Sinai Medical Center(西格拉姆山医疗中心)
Bridge the Gap between Language models and Tabular Understanding
Hong Kong University of Science and Technology (Guangzhou)(香港科技大学(广州));Hong Kong University of Science and Technology(香港科技大学);Microsoft(微软);Duke University(杜克大学);Yale University(耶鲁大学);Peking University(北京大学)
When Raw Data Prevails: Are Large Language Model Embeddings Effective in Numerical Data Representation for Medical Machine Learning Applications?
University of Colorado(科罗拉多大学);University of Wisconsin Madison(威斯康星大学麦迪逊分校);Mass General Brigham, Harvard Medical School(哈佛医学院麻省总医院);Boston Children Hospital, Harvard Medical School(哈佛医学院波士顿儿童医院);Loyola University Chicago(芝加哥洛约拉大学);Dana-Farber Cancer Institute(达纳-法伯癌症研究所)