一种基于多语言模型自监督对比学习的无参考自动语音识别质量度量方法
A Reference-less Quality Metric for Automatic Speech Recognition via Contrastive-Learning of a Multi-Language Model with Self-Supervision
AI总结：
提出一种基于多语言模型自监督对比学习的无参考ASR质量度量，无需真实转录即可评估ASR性能，与WER相关性高，集成时降低WER超7%。
摘要：
自动语音识别（ASR）系统质量评估的常见标准是基于参考的度量指标，如词错误率（WER），其计算依赖于人工标注的真实转录文本，而获取这些文本既耗时又昂贵。本文提出了一种多语言无参考质量度量方法，无需真实转录文本即可比较不同ASR模型在语音数据集上的性能。为了估计ASR假设的质量，我们采用自监督学习方式，通过对比学习对预训练语言模型（LM）进行微调。在多个由顶级商业ASR引擎在不同语言上生成的未见测试数据集上进行的实验中，所提出的无参考度量在所有实验中与WER分数及其排名的相关性均显著高于基于最先进多语言LM的困惑度度量，并且在用于假设集成时，WER降低了超过7%。微调模型和实验代码已公开，以支持可复现性：https://github.com/aixplain/NoRefER
英文摘要：
The common standard for quality evaluation of automatic speech recognition (ASR) systems is reference-based metrics such as the Word Error Rate (WER), computed using manual ground-truth transcriptions that are time-consuming and expensive to obtain. This work proposes a multi-language referenceless quality metric, which allows comparing the performance of different ASR models on a speech dataset without ground truth transcriptions. To estimate the quality of ASR hypotheses, a pre-trained language model (LM) is fine-tuned with contrastive learning in a self-supervised learning manner. In experiments conducted on several unseen test datasets consisting of outputs from top commercial ASR engines in various languages, the proposed referenceless metric obtains a much higher correlation with WER scores and their ranks than the perplexity metric from the state-of-art multi-lingual LM in all experiments, and also reduces WER by more than $7\%$ when used for ensembling hypotheses. The fine-tuned model and experiments are made available for the reproducibility: https://github.com/aixplain/NoRefER
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
