arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2205.05974cs.CL
收藏

多模态词分类的计算获取模型

A Computational Acquisition Model for Multimodal Word Categorization

The Hebrew University of Jerusalem(耶路撒冷希伯来大学);University of Melbourne(墨尔本大学)

Uri Berger, Gabriel Stanovsky, Omri Abend, Lea Frermann

摘要:

近年来,文本和图像的自监督建模技术的发展为儿童语言习得的计算模型提供了新的机会,而儿童语言习得被认为在很大程度上依赖于跨模态信号。然而,之前的研究受到了限制,因为它们依赖于在大型图像数据集上训练的视觉模型,这些图像数据集用一组预定义的描绘对象类别进行注释。这是(a)不忠实于儿童收到的信息和(b)禁止评估这些模型的类别学习任务,由于预先规定的类别结构。我们解决了这一差距,并提出了一个认知启发,多模态采集模型,训练从图像标题对自然的数据,使用跨模态自我监督。我们表明,该模型学习单词类别和对象识别能力,并提出了发展文献中报道的趋势。我们公开我们的代码和训练模型,以供将来参考和使用。

英文摘要:

Recent advances in self-supervised modeling of text and images open new opportunities for computational models of child language acquisition, which is believed to rely heavily on cross-modal signals. However, prior studies have been limited by their reliance on vision models trained on large image datasets annotated with a pre-defined set of depicted object categories. This is (a) not faithful to the information children receive and (b) prohibits the evaluation of such models with respect to category learning tasks, due to the pre-imposed category structure. We address this gap, and present a cognitively-inspired, multimodal acquisition model, trained from image-caption pairs on naturalistic data using cross-modal self-supervision. We show that the model learns word categories and object recognition abilities, and presents trends reminiscent of those reported in the developmental literature. We make our code and trained models public for future reference and use.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑