arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2603.23048cs.SDcs.AI
收藏

MSR-HuBERT:面向多种采样率的自监督预训练

MSR-HuBERT: Self-supervised Pre-training for Adaptation to Multiple Sampling Rates

Tianjin Key Laboratory of Cognitive Computing and Application(天津认知计算与应用重点实验室);Tianjin University(天津大学);Huiyan Technology Company, Ltd.(慧言科技有限公司);Chinese Academy of Sciences(中国科学院)

Zikang Huang, Meng Ge, Tianrui Wang, Xuanchen Li, Xiaobao Wang, Longbiao Wang, Jianwu Dang

AI总结:

本文提出MSRHuBERT,一种适应多种采样率的自监督预训练方法,通过改进HuBERT的下采样CNN,实现不同采样率原始波形到共享时间分辨率的映射,提升了语音识别和全频带语音重建性能。

摘要:

自监督学习(SSL)已推动语音处理的发展。然而,现有语音SSL方法通常假设单一采样率,难以处理混合采样率数据,因时间分辨率不匹配。为解决此问题,我们提出MSRHuBERT,一种多采样率自适应预训练方法。基于HuBERT,我们用一种多采样率自适应下采样CNN替代其单一率下采样CNN,将不同采样率的原始波形映射到共享时间分辨率而无需重采样。此设计实现了统一的混合采样率预训练和微调。在16至48kHz的实验中,MSRHuBERT在语音识别和全频带语音重建中优于HuBERT,保留高频细节的同时建模低频语义结构。此外,MSRHuBERT保留HuBERT的掩码预测目标和Transformer编码器,因此已有的HuBERT分析和改进可以直接应用。

英文摘要:

Self-supervised learning (SSL) has advanced speech processing. However, existing speech SSL methods typically assume a single sampling rate and struggle with mixed-rate data due to temporal resolution mismatch. To address this limitation, we propose MSRHuBERT, a multi-sampling-rate adaptive pre-training method. Building on HuBERT, we replace its single-rate downsampling CNN with a multi-sampling-rate adaptive downsampling CNN that maps raw waveforms from different sampling rates to a shared temporal resolution without resampling. This design enables unified mixed-rate pre-training and fine-tuning. In experiments spanning 16 to 48 kHz, MSRHuBERT outperforms HuBERT on speech recognition and full-band speech reconstruction, preserving high-frequency detail while modeling low-frequency semantic structure. Moreover, MSRHuBERT retains HuBERT's mask-prediction objective and Transformer encoder, so existing analyses and improvements that were developed for HuBERT can apply directly.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑