arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2406.09662cs.CLcs.AIcs.CV
收藏

通过接地学习语言结构

Learning Language Structures through Grounding

TOYOTA TECHNOLOGICAL INSTITUTE AT CHICAGO(芝加哥丰田技术研究所)

Freda Shi

AI总结:

本论文研究通过视觉、程序执行和其他语言等接地信号学习语言结构,提出视觉接地语法归纳、执行感知语义解析和跨语言结构投影等方法,显著提升解析、组合泛化及零样本跨语言性能。

摘要:

语言具有高度的结构性,其句法和语义结构在一定程度上被同一语言的使用者所共同认可。凭借对这些结构的隐式或显式意识,人类能够高效地学习和使用语言,并能泛化到包含未见过的单词的句子。受人类语言学习的启发,在本论文中,我们考虑了一类旨在通过接地学习语言结构的机器学习任务。我们从其他数据源(即接地信号)中寻求远程监督,包括但不限于其他模态(例如视觉)、程序的执行结果以及其他语言。我们通过三个方案展示了这种任务表述的潜力,并倡导其被采用。在第一部分中,我们考虑通过视觉接地学习句法解析。我们提出了视觉接地语法归纳任务,提出了首个从视觉接地文本和语音中归纳句法结构的模型,并发现视觉接地信号有助于提升解析质量,优于仅使用语言的模型。作为附带贡献,我们提出了一种新的评估指标,使得无需文本或自动语音识别系统即可评估语音解析。在第二部分中,我们提出了两种执行感知方法,将句子映射到相应的语义结构(即程序),显著提高了组合泛化能力和少样本程序合成能力。在第三部分中,我们提出了从其他语言的标注中学习语言结构的方法。具体而言,我们提出了一种在跨语言词对齐上达到新的最优水平的方法。随后,我们利用学习到的词对齐来提升零样本跨语言依存句法解析的性能,通过提出一种新颖的基于子结构的投影方法,保留从源语言学习到的结构知识。

英文摘要:

Language is highly structured, with syntactic and semantic structures, to some extent, agreed upon by speakers of the same language. With implicit or explicit awareness of such structures, humans can learn and use language efficiently and generalize to sentences that contain unseen words. Motivated by human language learning, in this dissertation, we consider a family of machine learning tasks that aim to learn language structures through grounding. We seek distant supervision from other data sources (i.e., grounds), including but not limited to other modalities (e.g., vision), execution results of programs, and other languages. We demonstrate the potential of this task formulation and advocate for its adoption through three schemes. In Part I, we consider learning syntactic parses through visual grounding. We propose the task of visually grounded grammar induction, present the first models to induce syntactic structures from visually grounded text and speech, and find that the visual grounding signals can help improve the parsing quality over language-only models. As a side contribution, we propose a novel evaluation metric that enables the evaluation of speech parsing without text or automatic speech recognition systems involved. In Part II, we propose two execution-aware methods to map sentences into corresponding semantic structures (i.e., programs), significantly improving compositional generalization and few-shot program synthesis. In Part III, we propose methods that learn language structures from annotations in other languages. Specifically, we propose a method that sets a new state of the art on cross-lingual word alignment. We then leverage the learned word alignments to improve the performance of zero-shot cross-lingual dependency parsing, by proposing a novel substructure-based projection method that preserves structural knowledge learned from the source language.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑