arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2109.01164eess.AScs.LGcs.SD
收藏

面向高质量大型语音数据集开发的可扩展数据标注流水线

Scalable Data Annotation Pipeline for High-Quality Large Speech Datasets Development

Appen

Mingkuan Liu, Chi Zhang, Hua Xing, Chao Feng, Monchu Chen, Judith Bishop, Grace Ngapo

摘要:

本文介绍了一个人在回路(HITL)数据注释管道,以生成高质量,大规模的语音数据集。该管道结合了人类和机器的优势,可以通过机器预标记和完全手动审计更快速,准确和经济高效地注释数据集。在注释管道中采用了盲测、行为监控和数据验证等质量控制机制,以减轻机器生成标签引入的潜在偏差。我们的A/B测试和试点结果表明,HITL管道可以将注释速度和容量提高至少80%,质量与手动双遍注释相当或更高。我们正在利用这个可扩展的管道来创建和不断增长用于多种语言的超高容量现成(UHV-OTS)语音语料库,每年每种语言的语音语料库扩展到10,000多小时。可以使用动态打包从UHV-OTS语料库生成定制数据集。UHV-OTS是一个长期的Appen项目,用于支持语音处理中的商业和学术研究数据需求。Appen每年将从UHV-OTS捐赠一些自由言论数据集,以支持CC-BY-SA许可下的学术和开源社区研究。我们还在Apache 2.0许可证下发布了数据预处理和预标记管道的代码,以允许复制论文中报告的结果。

英文摘要:

This paper introduces a human-in-the-loop (HITL) data annotation pipeline to generate high-quality, large-scale speech datasets. The pipeline combines human and machine advantages to more quickly, accurately, and cost-effectively annotate datasets with machine pre-labeling and fully manual auditing. Quality control mechanisms such as blind testing, behavior monitoring, and data validation have been adopted in the annotation pipeline to mitigate potential bias introduced by machine-generated labels. Our A/B testing and pilot results demonstrated the HITL pipeline can improve annotation speed and capacity by at least 80% and quality is comparable to or higher than manual double pass annotation. We are leveraging this scalable pipeline to create and continuously grow ultra-high volume off-the-shelf (UHV-OTS) speech corpora for multiple languages, with the capability to expand to 10,000+ hours per language annually. Customized datasets can be produced from the UHV-OTS corpora using dynamic packaging. UHV-OTS is a long-term Appen project to support commercial and academic research data needs in speech processing. Appen will donate a number of free speech datasets from the UHV-OTS each year to support academic and open source community research under the CC-BY-SA license. We are also releasing the code of the data pre-processing and pre-tagging pipeline under the Apache 2.0 license to allow reproduction of the results reported in the paper.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑