arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2406.03460eess.AScs.LGcs.SD
收藏

PESQetarian:论古德哈特定律与语音增强的相关性

The PESQetarian: On the Relevance of Goodhart's Law for Speech Enhancement

Danilo de Oliveira, Simon Welker, Julius Richter, Timo Gerkmann

AI总结:

本文通过提出在VB-DMD上PESQ达3.82但听感很差的“PESQetarian”模型,揭示了仅针对PESQ等单一指标优化语音增强模型会导致过拟合和误导性评估,主张结合多指标与主观听力测试。

摘要:

为了获得改进的语音增强模型,研究人员通常专注于根据特定仪器指标提高性能。然而,当同一指标被用于损失函数以优化模型时,它可能对该给定指标无法捕捉的方面造成损害。本文的目的是说明将语音增强模型过度拟合到用于评估的指标的风险。为此,我们引入了利用广泛使用的 PESQ 度量的增强模型。我们的“PESQetarian”模型在 VB-DMD 上达到了 3.82 的 PESQ,但在听力实验中得分非常低。虽然获得的 3.82 的 PESQ 值意味着在 VB-DMD 基准上具有“最先进的”PESQ 性能,但我们的示例表明,当针对某个指标进行优化时,仅在同一指标上进行孤立评估可能会产生误导。相反,评估中应包含其他指标,并且由此产生的性能预测应通过听力测试来确认。

英文摘要:

To obtain improved speech enhancement models, researchers often focus on increasing performance according to specific instrumental metrics. However, when the same metric is used in a loss function to optimize models, it may be detrimental to aspects that the given metric does not see. The goal of this paper is to illustrate the risk of overfitting a speech enhancement model to the metric used for evaluation. For this, we introduce enhancement models that exploit the widely used PESQ measure. Our "PESQetarian" model achieves 3.82 PESQ on VB-DMD while scoring very poorly in a listening experiment. While the obtained PESQ value of 3.82 would imply "state-of-the-art" PESQ-performance on the VB-DMD benchmark, our examples show that when optimizing w.r.t. a metric, an isolated evaluation on the same metric may be misleading. Instead, other metrics should be included in the evaluation and the resulting performance predictions should be confirmed by listening.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑