Recent advances in table understanding have focused on instruction-tuning large language models (LLMs) for table-related tasks. However, existing research has overlooked the impact of hyperparameter choices, and also lacks a comprehensive evaluation of the out-of-domain table understanding ability and the general capabilities of these table LLMs. In this paper, we evaluate these abilities in existing table LLMs, and find significant declines in both out-of-domain table understanding and general capabilities as compared to their base models. Through systematic analysis, we show that hyperparameters, such as learning rate, can significantly influence both table-specific and general capabilities. Contrary to the previous table instruction-tuning work, we demonstrate that smaller learning rates and fewer training instances can enhance table understanding while preserving general capabilities. Based on our findings, we introduce TAMA, a TAble LLM instruction-tuned from LLaMA 3.1 8B Instruct, which achieves performance on par with, or surpassing GPT-3.5 and GPT-4 on table tasks, while maintaining strong out-of-domain generalization and general capabilities. Our findings highlight the potential for reduced data annotation costs and more efficient model development through careful hyperparameter selection. We open-source the project and our models.
Rethinking the Instruction Quality: LIFT is What You Need
Shanghai Jiao Tong University(上海交通大学);Microsoft Cloud and AI(微软云与人工智能);Jilin University(吉林大学);Mohamed bin Zayed University of Artificial Intelligence(穆罕默德·本·扎耶德人工智能大学)
TableDreamer: Progressive and Weakness-guided Data Synthesis from Scratch for Table Instruction Tuning
Institute of Information Engineering, Chinese Academy of Sciences(中国科学院信息工程研究所);School of Cyber Security, University of Chinese Academy of Sciences(中国科学院大学网络空间安全学院);Baidu Inc(百度公司)
FollowTable: A Benchmark for Instruction-Following Table Retrieval
Southeast University(东南大学);Huawei Noah’s Ark Lab(华为诺亚实验室);Key Laboratory of New Generation Artificial Intelligence Technology and Its Interdisciplinary Applications (Southeast University), Ministry of Education(新一代人工智能技术及其交叉应用国家重点实验室(东南大学));Its Interdisciplinary Applications (Southeast University), Ministry of Education(交叉应用(东南大学),教育部)
Are Large Language Models Table-based Fact-Checkers?
Institute of Information Engineering, Chinese Academy of Sciences(中国科学院信息工程研究所);School of Cyber Security, University of Chinese Academy of Sciences(中国科学院大学网络空间安全学院)
Select2Reason: Efficient Instruction-Tuning Data Selection for Long-CoT Reasoning
DataArc Tech Ltd.(DataArc科技有限公司);Hong Kong University of Science and Technology (Guangzhou)(香港科学与技术大学(广州));IDEA Research, International Digital Economy Academy(IDEA研究所,国际数字经济学院);Hithink RoyalFlush Information Network Co., Ltd(Hithink RoyalFlush信息网络有限公司)
Rethinking Data Selection at Scale: Random Selection is Almost All You Need
Alibaba Group(阿里巴巴集团);School of Artificial Intelligence, Jilin University(吉林大学人工智能学院);International Center of Future Science, Jilin University(吉林大学未来科学国际中心)
TEAR: Table Extraction with Attribute Recommendation from Texts via Large Language Models
Hong Kong University of Science and Technology(香港科技大学);Hong Kong University of Science and Technology (Guangzhou)(香港科技大学(广州));South China Normal University(华南师范大学)
No Free Lunch: Rethinking Internal Feedback for LLM Reasoning
Zhongguancun Academy(中关村学院);Academy of Mathematics and Systems Science, Chinese Academy of Sciences(中国科学院数学与系统科学研究院);University of Chinese Academy of Sciences(中国科学院大学);Peking University(北京大学);Zhongguancun Institute of Artificial Intelligence(中关村人工智能研究院)
Institute of Information Engineering, Chinese Academy of Sciences(中国科学院信息工程研究所);Baidu Inc(百度公司);School of Artificial Intelligence, Beijing Normal University(北京师范大学人工智能学院);School of Cyber Security, University of Chinese Academy of Sciences(中国科学院大学网络空间安全学院)
Rethinking Table Pruning in TableQA: From Sequential Revisions to Gold Trajectory-Supervised Parallel Search
University of Science and Technology of China(中国科学技术大学);Institute of Artificial Intelligence, Hefei Comprehensive National Science Center(合肥综合国家科学中心人工智能研究院)
Rethinking Prompt Optimizers: From Prompt Merits to Optimization
School of Electrical and Electronic Engineering, Nanyang Technological University, Singapore(南洋理工大学电子与电气工程学院);Home Team Science and Technology Agency, Singapore(新加坡科技局)
Rethinking Heterogeneous LLM Merging: A Weighted Model Averaging Perspective
University of Science and Technology of China(中国科学技术大学);School of Computer Science and Technology, University of Science and Technology of China(中国科学技术大学计算机科学与技术学院)
"Set It Up!": Functional Object Arrangement with Compositional Generative Models
School of Computing, Smart System Institute, National University of Singapore(计算学院、智能系统研究所、新加坡国立大学);CSAIL, Massachusetts Institute of Technology(计算机科学与人工智能实验室、麻省理工学院)
Rethinking Stepwise Model Routing: A Cost-Efficient Table Reasoning Perspective
University of Science and Technology of China(中国科学技术大学);The University of Melbourne(墨尔本大学);Institute of Artificial Intelligence, Hefei Comprehensive National Science Center(合肥综合国家科学中心人工智能研究院)
Rethinking Data Curation in LLM Training: Online Reweighting Offers Better Generalization than Offline Methods
University of Cambridge(剑桥大学);OATML, University of Oxford(牛津大学OATML实验室);University of Toronto(多伦多大学);Shanghai Jiao Tong University(上海交通大学);Samsung AI Center(三星人工智能中心)
Rethinking Patient Education as Multi-turn Multi-modal Interaction
VA Bedford Health Care(VA贝德福德医疗中心);UMass Amherst(马萨诸塞大学阿默斯特分校);UMass Lowell(马萨诸塞大学洛厄尔分校);Yale University(耶鲁大学);National University of Singapore(新加坡国立大学);Yale School of Medicine(耶鲁医学院)
Using Large Language Models to Generate Clinical Trial Tables and Figures
Novartis Pharmaceuticals Corporation(诺华制药公司);School of Biomedical Informatics The University of Texas Health Science Center at Houston(德克萨斯大学休斯顿健康科学中心生物医学信息学院);Novartis Pharma AG(诺华制药公司);The University of Texas MD Anderson Cancer Center(德克萨斯大学 MD 安德森癌症中心)