多语言语音数据集的数据质量问题：对社会语言学意识和主动语言规划的需求
Data Quality Issues in Multilingual Speech Datasets: The Need for Sociolinguistic Awareness and Proactive Language Planning
AI总结：
本文审计了三个多语言语音数据集的质量问题，将其分为微观和宏观两类，发现宏观问题在资源不足语言中更普遍，通过台湾闽南语案例强调主动语言规划与质量控制的重要性，并提出改进建议。
摘要：
我们对三个广泛使用的公共多语言语音数据集——Mozilla Common Voice 17.0、FLEURS和Vox Populi——进行了质量审计，结果显示，在某些语言中，这些数据集存在严重的质量问题，这可能会掩盖下游评估结果，同时营造出一种成功的错觉。我们将这些质量问题分为两类：微观层面和宏观层面。我们发现，宏观层面的问题在制度化程度较低、通常资源不足的语言中更为普遍。我们对台湾闽南语（nan_tw）进行了案例分析，突显了在数据集创建过程中采取主动语言规划（如正字法规范、方言边界界定）和加强数据质量控制的必要性。最后，我们提出指导方针和建议，以在未来数据集开发中缓解这些问题，强调社会语言学意识和语言规划原则的重要性。此外，我们鼓励研究如何将这一创建过程本身作为社区主导的语言规划和复兴工具。
英文摘要：
Our quality audit for three widely used public multilingual speech datasets - Mozilla Common Voice 17.0, FLEURS, and Vox Populi - shows that in some languages, these datasets suffer from significant quality issues, which may obfuscate downstream evaluation results while creating an illusion of success. We divide these quality issues into two categories: micro-level and macro-level. We find that macro-level issues are more prevalent in less institutionalized, often under-resourced languages. We provide a case analysis of Taiwanese Southern Min (nan_tw) that highlights the need for proactive language planning (e.g. orthography prescriptions, dialect boundary definition) and enhanced data quality control in the dataset creation process. We conclude by proposing guidelines and recommendations to mitigate these issues in future dataset development, emphasizing the importance of sociolinguistic awareness and language planning principles. Furthermore, we encourage research into how this creation process itself can be leveraged as a tool for community-led language planning and revitalization.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
