arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 1806.06371cs.CLcs.AI
收藏

语言处理中的多模态Grounding

Multimodal Grounding for Language Processing

University of Duisburg-Essen(杜伊斯堡-埃森大学);Technische Universität Darmstadt(达姆施塔特工业大学)

Lisa Beinborn, Teresa Botschen, Iryna Gurevych

摘要:

本调查讨论了多模态处理的最新发展如何促进语言的概念基础。我们将多模态处理中的信息流与人类信息处理的认知模型进行了分类,并分析了不同的多模态表示方法。基于这种方法库存,我们讨论了多模态接地的各种语言处理任务的好处和出现的挑战。我们特别关注动词的多模态接地,这对语言的合成能力起着至关重要的作用。

英文摘要:

This survey discusses how recent developments in multimodal processing facilitate conceptual grounding of language. We categorize the information flow in multimodal processing with respect to cognitive models of human information processing and analyze different methods for combining multimodal representations. Based on this methodological inventory, we discuss the benefit of multimodal grounding for a variety of language processing tasks and the challenges that arise. We particularly focus on multimodal grounding of verbs which play a crucial role for the compositional power of language.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑