剖析设备上E2E语音识别的用户感知延迟
Dissecting User-Perceived Latency of On-Device E2E Speech Recognition
摘要：
随着智能手机和智能扬声器等语音设备变得越来越普遍，人们对构建可直接在设备上运行的自动语音识别（ASR）系统的兴趣越来越大;端到端（E2 E）语音识别模型，如递归神经网络换能器及其变体，最近已成为该任务的主要候选者。除了准确和紧凑之外，这样的系统还需要以低用户感知延迟（UPL）解码语音，一旦说出就产生单词。这项工作研究了各种技术-模型架构，训练标准，解码超参数和端点参数-对UPL的影响。我们的分析表明，模型大小的度量（参数，输入块大小）或计算的度量（例如，FLOPS，RTF）反映了模型处理输入帧的能力，并不总是与观察到的UPL密切相关。因此，传统的算法延迟测量可能不足以准确地捕获在嵌入式设备上部署模型时观察到的延迟。相反，我们发现影响令牌发射延迟和端点行为的因素对UPL有更大的影响。我们实现了延迟和字出错率之间的最佳权衡时，执行ASR联合端点，同时利用最近提出的对齐正则化机制。
英文摘要：
As speech-enabled devices such as smartphones and smart speakers become increasingly ubiquitous, there is growing interest in building automatic speech recognition (ASR) systems that can run directly on-device; end-to-end (E2E) speech recognition models such as recurrent neural network transducers and their variants have recently emerged as prime candidates for this task. Apart from being accurate and compact, such systems need to decode speech with low user-perceived latency (UPL), producing words as soon as they are spoken. This work examines the impact of various techniques - model architectures, training criteria, decoding hyperparameters, and endpointer parameters - on UPL. Our analyses suggest that measures of model size (parameters, input chunk sizes), or measures of computation (e.g., FLOPS, RTF) that reflect the model's ability to process input frames are not always strongly correlated with observed UPL. Thus, conventional algorithmic latency measurements might be inadequate in accurately capturing latency observed when models are deployed on embedded devices. Instead, we find that factors affecting token emission latency, and endpointing behavior have a larger impact on UPL. We achieve the best trade-off between latency and word error rate when performing ASR jointly with endpointing, while utilizing the recently proposed alignment regularization mechanism.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
