针对表格数据生成的大型语言模型指令微调——一天内完成
Instruction Tuning of Large Language Models for Tabular Data Generation-in One Day
AI总结：
本文提出了一种在有限资源下通过指令微调提升LLM表格数据生成能力的方法，利用高质量数据集和少量指令实现与GPT-4o相当的生成性能。
摘要：
表格指令微调已成为提升LLM对表格数据理解能力的一个有前途的研究方向。然而，现有的大多数工作只考虑了对表格数据进行问答和推理任务，忽略了表格数据生成。在本工作中，我们首次探索了指令微调在提升LLM表格数据生成能力方面的有效性。更具体地说，鉴于表格指令微调对数据和计算资源的需求很高，我们旨在探讨在有限的数据和计算资源下，通过指令微调实现表格数据生成的可能性。为此，我们首先创建了一个高质量的表格指令数据集，以实现高效的LLM理解。然后，我们使用该数据集的训练集对开源LLM（Llama3.1-8B-Instruct）进行指令微调，以提高其表格数据生成性能。我们的实验结果表明，通过使用我们高质量的数据集，并仅用7K条指令和A100 GPU在不到6小时内进行指令微调，我们实现了与最强大的商业LLM GPT-4o相当的表格数据生成性能。
英文摘要：
Tabular instruction tuning has emerged as a promising research direction for improving LLMs understanding of tabular data. However, the majority of existing works only consider question-answering and reasoning tasks over tabular data, leaving tabular data generation largely unnoticed. In this work, for the first time, we explore the efficacy of instruction tuning in improving LLMs tabular data generation capabilities. More specifically, given the high data and computation requirements of tabular instruction tuning, we aim to address the possibility of instruction tuning for tabular data generation with limited data and computational resources. To achieve this, we first create a high-quality instruction dataset for tabular data, enabling efficient LLM comprehension. We then instruction-tune an open-source LLM (Llama3.1-8B-Instruct) on the training set of this dataset to improve its tabular data generation performance. Our experimental results show that by using our high-quality dataset and instruction-tuning on only 7K instructions with an A100 GPU, for less than 6 hours, we achieve tabular data generation performance on par with the most capable commercial LLM, GPT-4o.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
