arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2401.15385cs.CLcs.MM
收藏

面向利用上下文线索从语音中进行事件抽取的研究

Towards Event Extraction from Speech with Contextual Clues

Southeast University(东南大学);Monash University(莫纳什大学)

Jingqi Kang, Tongtong Wu, Jinming Zhao, Guitao Wang, Guilin Qi, Yuan-Fang Li, Gholamreza Haffari

AI总结:

针对语音语义事件抽取探索不足的问题,本文提出SpeechEE任务,构建相关数据集,采用带上下文线索的序列到结构生成范式及扁平化事件表示,实现了显著的性能提升。

摘要:

尽管基于文本的事件抽取一直是活跃的研究领域,且已在诸多领域得到成功应用,但直接从语音中抽取语义事件仍是一个探索不足的问题。本文提出了语音事件抽取(SpeechEE)任务,构建了三个合成训练集和一个真人语音测试集。与文本事件抽取相比,SpeechEE面临更大挑战,主要原因在于语音信号是连续的、无语词边界的复杂信号。此外,与可感知的声音事件不同,语义事件更为微妙,需要更深层次的理解。为应对这些挑战,我们提出了一种序列到结构的生成范式,能够端到端地从语音信号中生成事件,还提出了一种条件生成方法,将语音识别转录文本用作上下文线索。我们进一步提出用扁平化格式表示事件,使输出更接近自然语言。实验结果表明,我们的方法在所有数据集上均带来显著提升,最高F1值提升达10.7%。代码和数据集已发布于https://github.com/jodie-kang/SpeechEE。

英文摘要:

While text-based event extraction has been an active research area and has seen successful application in many domains, extracting semantic events from speech directly is an under-explored problem. In this paper, we introduce the Speech Event Extraction (SpeechEE) task and construct three synthetic training sets and one human-spoken test set. Compared to event extraction from text, SpeechEE poses greater challenges mainly due to complex speech signals that are continuous and have no word boundaries. Additionally, unlike perceptible sound events, semantic events are more subtle and require a deeper understanding. To tackle these challenges, we introduce a sequence-to-structure generation paradigm that can produce events from speech signals in an end-to-end manner, together with a conditioned generation method that utilizes speech recognition transcripts as the contextual clue. We further propose to represent events with a flat format to make outputs more natural language-like. Our experimental results show that our method brings significant improvements on all datasets, achieving a maximum F1 gain of 10.7%. The code and datasets are released on https://github.com/jodie-kang/SpeechEE.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑