利用韵律注意力和蒸馏提升端到端SLU性能
Improving End-to-End SLU performance with Prosodic Attention and Distillation
AI总结：
针对端到端SLU忽略韵律信息的问题，提出韵律注意力和韵律蒸馏方法，在SLURP和STOP数据集上分别提升意图分类准确率8%和2%。
摘要：
大多数端到端SLU方法依赖预训练的ASR或语言模型特征进行意图预测。然而，语音中的其他重要信息，如韵律，常常被忽略。近期研究表明，通过融入韵律信息，对话行为分类的结果有所改善。这些方法的改进幅度很小，因为神经模型忽略了韵律特征。在这项工作中，我们提出了韵律注意力（prosody-attention），它利用韵律特征以不同方式生成话语时间帧上的注意力图。然后，我们提出了韵律蒸馏（prosody-distillation），以在声学编码器中显式学习韵律信息，而不是拼接隐式韵律特征。所提出的两种方法均改善了基线结果，其中韵律蒸馏方法在SLURP和STOP数据集上相比韵律基线，意图分类准确率分别提升了8%和2%。
英文摘要：
Most End-to-End SLU methods depend on the pretrained ASR or language model features for intent prediction. However, other essential information in speech, such as prosody, is often ignored. Recent research has shown improved results in classifying dialogue acts by incorporating prosodic information. The margins of improvement in these methods are minimal as the neural models ignore prosodic features. In this work, we propose prosody-attention, which uses the prosodic features differently to generate attention maps across time frames of the utterance. Then we propose prosody-distillation to explicitly learn the prosodic information in the acoustic encoder rather than concatenating the implicit prosodic features. Both the proposed methods improve the baseline results, and the prosody-distillation method gives an intent classification accuracy improvement of 8\% and 2\% on SLURP and STOP datasets over the prosody baseline.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
