基于深度神经网络的非监督听觉与语义协同适应模型
Unsupervised Auditory and Semantic Entrainment Models with Deep Neural Networks
AI总结:
该研究提出一种非监督深度学习框架,利用文本和听觉特征评估对话中的语义协同适应,模型能区分人-人与机器交互,且不同分析单元结果一致。
摘要:
说话者在对话中倾向于表现出适应性行为,即在与对话者交流时,在言语的多个方面变得相似,这种行为被称为协同适应(entrainment)。我们提出了一种非监督深度学习框架,该框架从文本特征中提取有意义的表征,用于开发语义协同适应模型。我们通过使用BERT模型的不同变体(DistilBERT和XLM-RoBERTa)以及谷歌通用句子编码器(USE)嵌入来提取特征,在两个人类-人类(HH)语料库(Fisher语料库英语第一部分、哥伦比亚游戏语料库)和一个人类-机器(HM)语料库(语音助手对话语料库(VACC))上研究了模型的性能。除了语义特征外,我们还训练了基于DNN的模型,利用两种听觉嵌入(TRIpLet损失网络(TRILL)向量、低级描述符(LLD)特征)和两种分析单元(停顿间单元和话轮)。结果表明,语义协同适应可以通过我们的模型进行评估,模型能够区分HH和HM交互,并且用于提取声学特征的两种分析单元提供了可比较的结果。
英文摘要:
Speakers tend to engage in adaptive behavior, known as entrainment, when they become similar to their interlocutor in various aspects of speaking. We present an unsupervised deep learning framework that derives meaningful representation from textual features for developing semantic entrainment. We investigate the model's performance by extracting features using different variations of the BERT model (DistilBERT and XLM-RoBERTa) and Google's universal sentence encoder (USE) embeddings on two human-human (HH) corpora (The Fisher Corpus English Part 1, Columbia games corpus) and one human-machine (HM) corpus (Voice Assistant Conversation Corpus (VACC)). In addition to semantic features we also trained DNN-based models utilizing two auditory embeddings (TRIpLet Loss network (TRILL) vectors, Low-level descriptors (LLD) features) and two units of analysis (Inter pausal unit and Turn). The results show that semantic entrainment can be assessed with our model, that models can distinguish between HH and HM interactions and that the two units of analysis for extracting acoustic features provide comparable findings.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。