arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2412.14890eess.AScs.SD
收藏

扩展这个,而非那个:探究用于高效语音增强扩展的关键数据集属性

Scale This, Not That: Investigating Key Dataset Attributes for Efficient Speech Enhancement Scaling

Shanghai Jiao Tong University(上海交通大学)

Leying Zhang, Wangyou Zhang, Chenda Li, Yanmin Qian

AI总结:

提出一种生成-训练-评估框架,利用零样本文本转语音系统受控合成数据,发现声学属性比语义属性对语音增强模型性能更重要。

摘要:

最近的语音增强模型通过扩大模型复杂度和训练数据展现出了显著的性能提升。然而,数据集变异性(例如文本、语言、说话人和噪声)的影响却鲜少被探索。单独分析每个属性通常具有挑战性,因为多个属性在常用数据集中通常相互交织,这构成了理解每个属性对模型性能独特贡献的重大障碍。为了应对这一挑战,我们提出了一种生成-训练-评估框架,利用零样本文本转语音系统来研究受控属性变化对语音增强性能的影响。它使我们能够以可扩展的方式合成训练数据集,同时仔细改变每个属性。基于所提出的框架,我们分析了各种数据集属性的缩放效应对判别式和生成式SE模型性能的影响。在多领域语料库上的大量实验表明,声学属性(例如说话人和噪声)对当前的语音增强模型比语义属性(例如语言和文本)重要得多,为未来的研究提供了新见解。

英文摘要:

Recent speech enhancement models have shown impressive performance gains by scaling up model complexity and training data. However, the impact of dataset variability (e.g. text, language, speaker, and noise) has been underexplored. Analyzing each attribute individually is often challenging, as multiple attributes are usually entangled in commonly used datasets, posing a significant obstacle in understanding the distinct contributions of each attribute to the model's performance. To address this challenge, we propose a generation-training-evaluation framework that leverages zero-shot text-to-speech systems to investigate the impact of controlled attribute variations on speech enhancement performance. It enables us to synthesize training datasets in a scalable manner while carefully altering each attribute. Based on the proposed framework, we analyze the scaling effects of various dataset attributes on the performance of both discriminative and generative SE models. Extensive experiments on multi-domain corpora imply that acoustic attributes (e.g., speaker and noise) are much more important to current speech enhancement models than semantic attributes (e.g., language and text), offering new insights for future research.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑