MSR-HuBERT:面向多种采样率的自监督预训练
MSR-HuBERT: Self-supervised Pre-training for Adaptation to Multiple Sampling Rates
AI总结:
本文提出MSRHuBERT,一种适应多种采样率的自监督预训练方法,通过改进HuBERT的下采样CNN,实现不同采样率原始波形到共享时间分辨率的映射,提升了语音识别和全频带语音重建性能。
摘要:
自监督学习(SSL)已推动语音处理的发展。然而,现有语音SSL方法通常假设单一采样率,难以处理混合采样率数据,因时间分辨率不匹配。为解决此问题,我们提出MSRHuBERT,一种多采样率自适应预训练方法。基于HuBERT,我们用一种多采样率自适应下采样CNN替代其单一率下采样CNN,将不同采样率的原始波形映射到共享时间分辨率而无需重采样。此设计实现了统一的混合采样率预训练和微调。在16至48kHz的实验中,MSRHuBERT在语音识别和全频带语音重建中优于HuBERT,保留高频细节的同时建模低频语义结构。此外,MSRHuBERT保留HuBERT的掩码预测目标和Transformer编码器,因此已有的HuBERT分析和改进可以直接应用。
英文摘要:
Self-supervised learning (SSL) has advanced speech processing. However, existing speech SSL methods typically assume a single sampling rate and struggle with mixed-rate data due to temporal resolution mismatch. To address this limitation, we propose MSRHuBERT, a multi-sampling-rate adaptive pre-training method. Building on HuBERT, we replace its single-rate downsampling CNN with a multi-sampling-rate adaptive downsampling CNN that maps raw waveforms from different sampling rates to a shared temporal resolution without resampling. This design enables unified mixed-rate pre-training and fine-tuning. In experiments spanning 16 to 48 kHz, MSRHuBERT outperforms HuBERT on speech recognition and full-band speech reconstruction, preserving high-frequency detail while modeling low-frequency semantic structure. Moreover, MSRHuBERT retains HuBERT's mask-prediction objective and Transformer encoder, so existing analyses and improvements that were developed for HuBERT can apply directly.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。