基于共现线索的双重分节分析的无监督多模态词汇发现
Unsupervised Multimodal Word Discovery based on Double Articulation Analysis with Co-occurrence cues
AI总结：
本文提出融合 NPB-DAA 与 MLDA 的全无监督多模态方法，利用音系分布及视觉、触觉、听觉共现线索从语音中发现词语和音素，实验证明其词汇发现与切分性能优于基线。
摘要：
人类婴儿在几乎不具备语言先验知识的情况下，依据音系分布的统计特性以及其他感觉刺激的共现关系习得口头词汇。本研究提出了一种新颖的全无监督学习方法，用于发现语音单元：该方法将音系信息作为分布线索，将物体信息作为共现线索。所提方法能够通过无监督学习从语音信号中习得词语和音素，并同时利用来自视觉、触觉和听觉等多种模态的物体信息。该方法基于非参数贝叶斯双重分节分析器（NPB-DAA）从音系特征中发现音素和词语，并采用多模态潜在 Dirichlet 分配（MLDA）对从物体获得的多模态信息进行分类。在实验中，所提方法表现出高于基线方法的词汇发现性能；表达物体特征的词语（即对应名词和形容词的词语）被准确切分。此外，作者还考察了语言信息重要性的差异如何影响学习性能。结果表明，与固定条件相比，增大词语模态的权重进一步提升了性能。
英文摘要：
Human infants acquire their verbal lexicon with minimal prior knowledge of language based on the statistical properties of phonological distributions and the co-occurrence of other sensory stimuli. This study proposes a novel fully unsupervised learning method for discovering speech units using phonological information as a distributional cue and object information as a co-occurrence cue. The proposed method can acquire words and phonemes from speech signals using unsupervised learning and utilize object information based on multiple modalities-vision, tactile, and auditory-simultaneously. The proposed method is based on the nonparametric Bayesian double articulation analyzer (NPB-DAA) discovering phonemes and words from phonological features, and multimodal latent Dirichlet allocation (MLDA) categorizing multimodal information obtained from objects. In an experiment, the proposed method showed higher word discovery performance than baseline methods. Words that expressed the characteristics of objects (i.e., words corresponding to nouns and adjectives) were segmented accurately. Furthermore, we examined how learning performance is affected by differences in the importance of linguistic information. Increasing the weight of the word modality further improved performance relative to that of the fixed condition.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
