揭示与提升数据可信度：基于训练无害语言模型数据集的研究
Unmasking and Improving Data Credibility: A Study with Datasets for Training Harmless Language Models
AI总结：
本研究提出一个系统框架评估真实世界数据集的可信度，识别并修复标签错误，显著提升数据可信度与下游学习性能，并开源了数据清洗工具Docta。
摘要：
语言模型在各类任务中展现出潜力，但在训练、微调或对齐过程中可能受到不良数据的影响。例如，若部分不安全对话被错误标注为安全，基于这些样本微调的模型可能产生危害。因此，标注的正确性，即数据集的可信度至关重要。本研究聚焦于真实世界数据集的可信度，包括可用于训练无害语言模型的流行基准：Jigsaw Civil Comments、Anthropic Harmless & Red Team、PKU BeaverTails & SafeRLHF。鉴于人工清洗这些数据集的成本与难度，我们引入了一个系统框架，用于评估数据集可信度、识别标签错误，并评估精选语言数据中噪声标签的影响，特别聚焦于不安全评论和对话分类。利用该框架，我们在从上述基准构建的11个数据集中发现并修复了平均6.16%的标签错误。通过直接修复标签错误，数据可信度和下游学习性能得到显著提升，表明了清洗现有真实世界数据集的重要性。我们提供了开源工具Docta用于数据清洗，网址为https://github.com/Docta-ai/docta。
英文摘要：
Language models have shown promise in various tasks but can be affected by undesired data during training, fine-tuning, or alignment. For example, if some unsafe conversations are wrongly annotated as safe ones, the model fine-tuned on these samples may be harmful. Therefore, the correctness of annotations, i.e., the credibility of the dataset, is important. This study focuses on the credibility of real-world datasets, including the popular benchmarks Jigsaw Civil Comments, Anthropic Harmless & Red Team, PKU BeaverTails & SafeRLHF, that can be used for training a harmless language model. Given the cost and difficulty of cleaning these datasets by humans, we introduce a systematic framework for evaluating the credibility of datasets, identifying label errors, and evaluating the influence of noisy labels in the curated language data, specifically focusing on unsafe comments and conversation classification. With the framework, we find and fix an average of 6.16% label errors in 11 datasets constructed from the above benchmarks. The data credibility and downstream learning performance can be remarkably improved by directly fixing label errors, indicating the significance of cleaning existing real-world datasets. We provide an open-source tool, Docta, for data cleaning at https://github.com/Docta-ai/docta.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
