大语言模型能否增强低资源阅读理解数据集？机遇与挑战
Can LLMs Augment Low-Resource Reading Comprehension Datasets? Opportunities and Challenges
AI总结：
该研究探究GPT-4能否增强低资源抽取式阅读理解数据集，通过对比微调性能与标注成本评估其替代人工标注的效果，分析了LLMs作为问答系统合成数据增强器的机遇挑战，并发布增强版数据集供研究使用。
摘要：
大语言模型（LLMs）已在广泛的自然语言处理（NLP）任务中展现出出色的零样本性能，具备推理与运用常识的能力。一个相关应用是利用它们为下游任务创建高质量的合成数据集。本研究探究了GPT-4是否可用于增强现有的抽取式阅读理解数据集。自动化数据标注流程有望节省手动标注数据集所耗费的大量时间、资金与人力。本文通过比较微调后的性能以及标注相关成本，评估了GPT-4在低资源阅读理解任务中替代人类标注者的表现。这项工作是首个针对LLMs作为问答系统合成数据增强器的分析，凸显了其独特的机遇与挑战。此外，我们发布了低资源数据集的增强版本，以供研究界构建更多评估生成数据集的基准。
英文摘要：
Large Language Models (LLMs) have demonstrated impressive zero shot performance on a wide range of NLP tasks, demonstrating the ability to reason and apply commonsense. A relevant application is to use them for creating high quality synthetic datasets for downstream tasks. In this work, we probe whether GPT-4 can be used to augment existing extractive reading comprehension datasets. Automating data annotation processes has the potential to save large amounts of time, money and effort that goes into manually labelling datasets. In this paper, we evaluate the performance of GPT-4 as a replacement for human annotators for low resource reading comprehension tasks, by comparing performance after fine tuning, and the cost associated with annotation. This work serves to be the first analysis of LLMs as synthetic data augmenters for QA systems, highlighting the unique opportunities and challenges. Additionally, we release augmented versions of low resource datasets, that will allow the research community to create further benchmarks for evaluation of generated datasets.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
