多分辨率HuBERT的探索
Exploration on HuBERT with Multiple Resolutions
AI总结:
针对HuBERT固定20ms分辨率不适应不同时间尺度语音任务的问题,提出并行与层次两种多分辨率整合方法,实验证明其性能优于原始模型。
摘要:
隐藏单元BERT(HuBERT)是一种在语音处理中广泛使用的自监督学习(SSL)模型。然而,我们认为其固定的20ms隐藏表示分辨率对于各种语音处理任务并非最优,因为这些任务的属性(例如说话人特征和语义)基于不同的时间尺度。为解决这一局限,我们提出在下游任务中使用多分辨率的HuBERT表示。我们探索了两种方法,即并行方法和层次方法,用于整合不同分辨率的HuBERT特征。通过实验,我们证明多分辨率HuBERT优于原始模型。这凸显了在类似HuBERT的SSL模型中利用多分辨率来捕获语音信号中多样化信息的潜力。
英文摘要:
Hidden-unit BERT (HuBERT) is a widely-used self-supervised learning (SSL) model in speech processing. However, we argue that its fixed 20ms resolution for hidden representations would not be optimal for various speech-processing tasks since their attributes (e.g., speaker characteristics and semantics) are based on different time scales. To address this limitation, we propose utilizing HuBERT representations at multiple resolutions for downstream tasks. We explore two approaches, namely the parallel and hierarchical approaches, for integrating HuBERT features with different resolutions. Through experiments, we demonstrate that HuBERT with multiple resolutions outperforms the original model. This highlights the potential of utilizing multiple resolutions in SSL models like HuBERT to capture diverse information from speech signals.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。