arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2110.06263cs.CLcs.AIcs.SDeess.AS
收藏

基于限制自注意的语音摘要

Speech Summarization using Restricted Self-Attention

Carnegie Mellon University(卡内基梅隆大学)

Roshan Sharma, Shruti Palaskar, Alan W Black, Florian Metze

摘要:

语音摘要通常通过使用级联的语音识别和文本摘要模型来执行。语音摘要模型的端到端建模是具有挑战性的,这是由于长输入音频序列引起的存储器和计算约束。最近在文档摘要方面的工作启发了一些方法来降低自我关注的复杂性,这使得Transformer模型能够处理长序列。在这项工作中,我们介绍了一个单一的模型优化端到端的语音摘要。我们将限制性自注意技术从基于文本的模型应用到语音模型,以解决内存和计算限制。我们证明,该模型学习直接总结语音教学视频的How-2语料库。所提出的端到端模型优于先前提出的级联模型3点绝对的ROUGE。此外,我们认为口语语言理解任务预测概念从语音输入,并表明所提出的端到端模型优于级联模型的4点绝对F-1。

英文摘要:

Speech summarization is typically performed by using a cascade of speech recognition and text summarization models. End-to-end modeling of speech summarization models is challenging due to memory and compute constraints arising from long input audio sequences. Recent work in document summarization has inspired methods to reduce the complexity of self-attentions, which enables transformer models to handle long sequences. In this work, we introduce a single model optimized end-to-end for speech summarization. We apply the restricted self-attention technique from text-based models to speech models to address the memory and compute constraints. We demonstrate that the proposed model learns to directly summarize speech for the How-2 corpus of instructional videos. The proposed end-to-end model outperforms the previously proposed cascaded model by 3 points absolute on ROUGE. Further, we consider the spoken language understanding task of predicting concepts from speech inputs and show that the proposed end-to-end model outperforms the cascade model by 4 points absolute F-1.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑