可视化分析上下文嵌入
Visually Analyzing Contextualized Embeddings
摘要：
在本文中，我们介绍了一种可视化分析基于深度神经网络的语言模型产生的上下文化嵌入的方法。我们的方法受到了自然语言处理的语言探测器的启发，其中任务被设计为探测语言结构的语言模型，例如词性和命名实体。然而，这些方法在很大程度上是验证性的，仅使用户能够测试先验已知的信息。在这项工作中，我们避免监督探测任务，并提倡无监督探测，加上视觉探索技术，以评估语言模型学到了什么。具体来说，我们集群上下文嵌入产生的大型文本语料库，并介绍了基于此聚类和文本结构的可视化设计-集群同现，集群跨度，集群词成员资格-以帮助引出的功能，和之间的关系，个别集群。用户反馈突出了我们的设计在发现不同类型的语言结构方面的优势。
英文摘要：
In this paper we introduce a method for visually analyzing contextualized embeddings produced by deep neural network-based language models. Our approach is inspired by linguistic probes for natural language processing, where tasks are designed to probe language models for linguistic structure, such as parts-of-speech and named entities. These approaches are largely confirmatory, however, only enabling a user to test for information known a priori. In this work, we eschew supervised probing tasks, and advocate for unsupervised probes, coupled with visual exploration techniques, to assess what is learned by language models. Specifically, we cluster contextualized embeddings produced from a large text corpus, and introduce a visualization design based on this clustering and textual structure - cluster co-occurrences, cluster spans, and cluster-word membership - to help elicit the functionality of, and relationship between, individual clusters. User feedback highlights the benefits of our design in discovering different types of linguistic structures.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
