语标信息有助于学习更好的自然语言推理表示
Logographic Information Aids Learning Better Representations for Natural Language Inference
摘要：
统计语言模型通常基于词或其他形式单元的上下文分布来实现表示学习，而与书面文本的语标特征相关的任何信息通常被忽略，假设它们应该依赖于共现统计来检索。另一方面，随着语言模型变得更大并且需要更多的数据来学习可靠的表示，这样的假设可能开始回落，特别是在数据稀疏的条件下。许多语言，包括汉语和越南语，使用语标书写系统，其中表面形式表示为较小的字形单位的视觉组织，这些单位通常包含许多语义线索。在本文中，我们提出了一个新的研究，探讨了提供语言模型与语标信息，学习更好的语义表示的好处。我们测试我们的假设，在自然语言推理（NLI）的任务，通过评估的好处计算多模态表示，结合上下文信息与语义信息。我们在六种不同类型和书写系统的语言中的评估结果表明，在具有logograhic系统的语言中使用多模态嵌入具有显着的好处，特别是对于出现统计较少的单词。
英文摘要：
Statistical language models conventionally implement representation learning based on the contextual distribution of words or other formal units, whereas any information related to the logographic features of written text are often ignored, assuming they should be retrieved relying on the cooccurence statistics. On the other hand, as language models become larger and require more data to learn reliable representations, such assumptions may start to fall back, especially under conditions of data sparsity. Many languages, including Chinese and Vietnamese, use logographic writing systems where surface forms are represented as a visual organization of smaller graphemic units, which often contain many semantic cues. In this paper, we present a novel study which explores the benefits of providing language models with logographic information in learning better semantic representations. We test our hypothesis in the natural language inference (NLI) task by evaluating the benefit of computing multi-modal representations that combine contextual information with glyph information. Our evaluation results in six languages with different typology and writing systems suggest significant benefits of using multi-modal embeddings in languages with logograhic systems, especially for words with less occurence statistics.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
