arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2404.09682cs.CLcs.AI
收藏

Multi-News+:基于LLM数据标注的高性价比数据集清洗

Multi-News+: Cost-efficient Dataset Cleansing via LLM-based Data Annotation

Chung-Ang University(中央大学);Graduate School of Advanced Imaging Sciences, Multimedia and Film, Chung-Ang University(中央大学高级影像科学、多媒体与电影研究生院)

Juhwan Choi, Jungmin Yun, Kyohoon Jin, YoungBin Kim

AI总结:

本研究利用大语言模型结合思维链和多数投票策略,对多文档摘要数据集Multi-News进行清洗以剔除不相关文档,构建了增强版Multi-News+,提供了一种高效且低成本的数据集质量提升方法。

摘要:

数据集质量对于确保下游任务模型的最佳性能和可靠性至关重要。然而,数据集在构建过程中常会无意混入噪声数据。以往多通过人工标注员纠正此问题,但雇佣和管理人工标注员既昂贵又耗时。作为替代,近期研究开始探索利用大语言模型(LLMs)进行数据标注。本研究提出案例,将基于LLM的数据标注扩展应用于通过清洗策略提升现有数据集质量。具体而言,我们利用思维链和多数投票等方法模仿人工标注,从多文档摘要任务广泛使用的Multi-News数据集中分类出不相关的文档。通过所提清洗方法,我们引入了增强版Multi-News+。通过采用LLMs进行数据清洗,我们展示了一种高效且有效的方法,无需依赖昂贵的人工标注即可提升数据集质量。

英文摘要:

The quality of the dataset is crucial for ensuring optimal performance and reliability of downstream task models. However, datasets often contain noisy data inadvertently included during the construction process. Numerous attempts have been made to correct this issue through human annotators. However, hiring and managing human annotators is expensive and time-consuming. As an alternative, recent studies are exploring the use of large language models (LLMs) for data annotation. In this study, we present a case study that extends the application of LLM-based data annotation to enhance the quality of existing datasets through a cleansing strategy. Specifically, we leverage approaches such as chain-of-thought and majority voting to imitate human annotation and classify unrelated documents from the Multi-News dataset, which is widely used for the multi-document summarization task. Through our proposed cleansing method, we introduce an enhanced Multi-News+. By employing LLMs for data cleansing, we demonstrate an efficient and effective approach to improving dataset quality without relying on expensive human annotation efforts.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑