arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2501.14693cs.CLcs.AI
收藏

重新思考表格指令微调

Rethinking Table Instruction Tuning

University of Michigan(密歇根大学)

Naihao Deng, Rada Mihalcea

AI总结:

本文重新审视表格指令微调,发现现有模型在域外表格理解和通用能力上显著下降,并证明较小学习率和较少训练实例可兼顾两者,由此提出TAMA模型,性能媲美GPT-3.5/4,并开源。

摘要:

近期表格理解领域的进展集中于针对表格相关任务对大型语言模型(LLM)进行指令微调。然而,现有研究忽视了超参数选择的影响,并且缺乏对表格LLM的域外表格理解能力和通用能力的全面评估。在本文中,我们评估了现有表格LLM的这些能力,并发现与它们的基础模型相比,域外表格理解和通用能力均显著下降。通过系统分析,我们表明超参数(如学习率)能显著影响表格特定能力和通用能力。与之前的表格指令微调工作相反,我们证明较小的学习率和较少的训练实例可以增强表格理解,同时保持通用能力。基于我们的发现,我们引入了TAMA,一个从LLaMA 3.1 8B Instruct指令微调得到的表格LLM,它在表格任务上取得了与GPT-3.5和GPT-4相当或超越的性能,同时保持了强大的域外泛化能力和通用能力。我们的发现强调了通过仔细选择超参数来降低数据标注成本和实现更高效模型开发的潜力。我们开源了项目和模型。

英文摘要:

Recent advances in table understanding have focused on instruction-tuning large language models (LLMs) for table-related tasks. However, existing research has overlooked the impact of hyperparameter choices, and also lacks a comprehensive evaluation of the out-of-domain table understanding ability and the general capabilities of these table LLMs. In this paper, we evaluate these abilities in existing table LLMs, and find significant declines in both out-of-domain table understanding and general capabilities as compared to their base models. Through systematic analysis, we show that hyperparameters, such as learning rate, can significantly influence both table-specific and general capabilities. Contrary to the previous table instruction-tuning work, we demonstrate that smaller learning rates and fewer training instances can enhance table understanding while preserving general capabilities. Based on our findings, we introduce TAMA, a TAble LLM instruction-tuned from LLaMA 3.1 8B Instruct, which achieves performance on par with, or surpassing GPT-3.5 and GPT-4 on table tasks, while maintaining strong out-of-domain generalization and general capabilities. Our findings highlight the potential for reduced data annotation costs and more efficient model development through careful hyperparameter selection. We open-source the project and our models.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑