arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2602.15519eess.AScs.SD
收藏

Enroll-on-Wakeup:真实噪声人机对话场景中无缝交互的目标语音提取首次比较研究

Enroll-on-Wakeup: A First Comparative Study of Target Speech Extraction for Seamless Interaction in Real Noisy Human-Machine Dialogue Scenarios

Shanghai Normal University(上海师范大学);Unisound AI Technology Co., Ltd.(Unisound人工智能技术有限公司)

Yiming Yang, Guangyong Wang, Haixin Guan, Yanhua Long

AI总结:

提出Enroll-on-Wakeup框架,利用唤醒词片段作为注册参考,无需预录语音,实现无缝交互;首次系统比较了判别式和生成式模型在真实噪声条件下的性能,并探索了基于LLM的TTS注册增强。

摘要:

目标语音提取(TSE)通常依赖于预先录制的高质量注册语音,这破坏了用户体验并限制了在自发交互中的可行性。在本文中,我们提出了Enroll-on-Wakeup(EoW),一种新颖的框架,其中在人机交互过程中自然捕获的唤醒词片段被自动用作注册参考。这消除了对预收集语音的需求,以实现无缝体验。我们首次对EoW-TSE进行了系统研究,评估了在真实多样声学条件下的先进判别式和生成式模型。鉴于唤醒词片段的短时和噪声特性,我们研究了使用基于LLM的TTS进行注册增强。结果表明,虽然当前的TSE模型在EoW-TSE中面临性能下降,但基于TTS的辅助显著增强了听觉体验,尽管在语音识别准确性方面仍存在差距。

英文摘要:

Target speech extraction (TSE) typically relies on pre-recorded high-quality enrollment speech, which disrupts user experience and limits feasibility in spontaneous interaction. In this paper, we propose Enroll-on-Wakeup (EoW), a novel framework where the wake-word segment, captured naturally during human-machine interaction, is automatically utilized as the enrollment reference. This eliminates the need for pre-collected speech to enable a seamless experience. We perform the first systematic study of EoW-TSE, evaluating advanced discriminative and generative models under real diverse acoustic conditions. Given the short and noisy nature of wake-word segments, we investigate enrollment augmentation using LLM-based TTS. Results show that while current TSE models face performance degradation in EoW-TSE, TTS-based assistance significantly enhances the listening experience, though gaps remain in speech recognition accuracy.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑