arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2601.04508cs.CLcs.AIcs.SD
收藏

WESR: 词级事件-语音识别的扩展与评估

WESR: Scaling and Evaluating Word-level Event-Speech Recognition

Fudan University(复旦大学);Shanghai Innovation Institute(上海创新研究院)

Chenchen Yang, Kexin Huang, Liwei Fan, Qian Tu, Botian Jiang, Dong Zhang, Linqi Yin, Shimin Li, Zhaoye Fei, Qinyuan Cheng, Xipeng Qiu

AI总结:

WESR提出词级事件-语音识别的扩展与评估框架,通过细化分类和专家标注的评估集,实现对非语言事件的精确定位,提升语音识别与处理的准确性。

摘要:

语音不仅传达语言信息,还包含丰富的非语言声学事件,如笑声和哭泣。尽管语义转录已得到充分研究,但非语言事件的精确定位仍然是一个关键但研究不足的挑战。当前方法存在任务定义不足、类别覆盖有限和时间粒度模糊的问题。它们还缺乏标准化的评估框架,阻碍了下游应用的发展。为此,我们首先开发了21种语音事件的细化分类法,将其新分类为离散(独立)与连续(混合语音)类型。基于细化的分类法,我们引入WESR-Bench,一个专家标注的评估集(900多个语音片段),具有新的位置感知协议,能够分离ASR错误与事件检测,从而实现对离散和连续事件的精确定位测量。我们还通过构建1700多个小时的语料库,并训练专用模型,超越了开源音频-语言模型和商业API,同时保持了ASR质量。我们预计WESR将成为未来研究丰富、真实世界听觉场景建模的基础资源。

英文摘要:

Speech conveys not only linguistic information but also rich non-verbal vocal events such as laughing and crying. While semantic transcription is well-studied, the precise localization of non-verbal events remains a critical yet under-explored challenge. Current methods suffer from insufficient task definitions with limited category coverage and ambiguous temporal granularity. They also lack standardized evaluation frameworks, hindering the development of downstream applications. To bridge this gap, we first develop a refined taxonomy of 21 vocal events, with a new categorization into discrete (standalone) versus continuous (mixed with speech) types. Based on the refined taxonomy, we introduce WESR-Bench, an expert-annotated evaluation set (900+ utterances) with a novel position-aware protocol that disentangles ASR errors from event detection, enabling precise localization measurement for both discrete and continuous events. We also build a strong baseline by constructing a 1,700+ hour corpus, and train specialized models, surpassing both open-source audio-language models and commercial APIs while preserving ASR quality. We anticipate that WESR will serve as a foundational resource for future research in modeling rich, real-world auditory scenes.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑