一个探测器和一个解析器的故事
A Tale of a Probe and a Parser
摘要：
测量语言的神经模型中编码的语言信息在NLP中已经变得流行。研究人员通过训练“探针”来实现这一目标-监督模型旨在从另一个模型的输出中提取语言结构。其中一种探针是结构探针（Hewitt和Manning，2019），旨在量化句法信息在上下文单词表征中编码的程度。结构探针有一个新颖的设计，在解析文献中没有得到证实，其确切的好处并不明显。为了探讨是否句法探测器会更好地利用现有的技术，我们比较结构探测器与一个更传统的解析器具有相同的轻量级参数化。在九种分析语言中的七种中，解析器在UUAS上的性能优于结构探测器，通常是相当大的一部分（例如，英语中的11.1分）。然而，在第二个不太常见的度量标准下，存在相反的趋势-结构探测器优于解析器。这就引出了一个问题：我们应该选择哪种度量标准？
英文摘要：
Measuring what linguistic information is encoded in neural models of language has become popular in NLP. Researchers approach this enterprise by training "probes" - supervised models designed to extract linguistic structure from another model's output. One such probe is the structural probe (Hewitt and Manning, 2019), designed to quantify the extent to which syntactic information is encoded in contextualised word representations. The structural probe has a novel design, unattested in the parsing literature, the precise benefit of which is not immediately obvious. To explore whether syntactic probes would do better to make use of existing techniques, we compare the structural probe to a more traditional parser with an identical lightweight parameterisation. The parser outperforms structural probe on UUAS in seven of nine analysed languages, often by a substantial amount (e.g. by 11.1 points in English). Under a second less common metric, however, there is the opposite trend - the structural probe outperforms the parser. This begs the question: which metric should we prefer?
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
