面向在线与离线场景的端到端多说话人语音识别实用性能改进
Improving Practical Aspects of End-to-End Multi-Talker Speech Recognition for Online and Offline Scenarios
AI总结：
本研究扩展序列化输出训练框架，通过结合连续语音分离前端、双模型/双遍架构及分段SOT，平衡多说话人语音识别的延迟与准确率，适配在线与离线场景需求。
摘要：
我们扩展了序列化输出训练（Serialized Output Training，SOT）框架，以满足流式与离线自动语音识别（automatic speech recognition，ASR）应用的实际需求。该方法聚焦于平衡延迟与准确率，适配实时字幕生成与摘要需求。我们提出了几项关键改进：（1）将连续语音分离（Continuous Speech Separation，CSS）单声道前端与端到端（end-to-end，E2E）系统结合用于高度重叠场景，打破了E2E与级联架构相对立的传统认知。CSS框架通过分离多说话人的重叠语音，提升ASR系统的准确率。（2）实现双模型架构——用于流式场景的Conformer Transducer与用于离线场景的序列到序列模型，或采用基于级联编码器的双遍模型。（3）探索更适配离线场景的基于分段的SOT（segSOT），同时提升多说话人转录文本的可读性。
英文摘要：
We extend the frameworks of Serialized Output Training (SOT) to address practical needs of both streaming and offline automatic speech recognition (ASR) applications. Our approach focuses on balancing latency and accuracy, catering to real-time captioning and summarization requirements. We propose several key improvements: (1) Leveraging Continuous Speech Separation (CSS) single-channel front-end with end-to-end (E2E) systems for highly overlapping scenarios, challenging the conventional wisdom of E2E versus cascaded setups. The CSS framework improves the accuracy of the ASR system by separating overlapped speech from multiple speakers. (2) Implementing dual models -- Conformer Transducer for streaming and Sequence-to-Sequence for offline -- or alternatively, a two-pass model based on cascaded encoders. (3) Exploring segment-based SOT (segSOT) which is better suited for offline scenarios while also enhancing readability of multi-talker transcriptions.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
