无监督神经语法归纳中的结构优化歧义与简单性偏置
Structural Optimization Ambiguity and Simplicity Bias in Unsupervised Neural Grammar Induction
AI总结：
本文分析了无监督神经语法归纳中的结构优化歧义与简单性偏置问题，提出逐句分析聚焦方法，利用预训练解析器的结构偏置减少损失评估的分析池，显著提升了性能并降低了方差与简单性偏置。
摘要：
神经参数化显著推进了无监督语法归纳。然而，使用传统似然损失对所有可能的分析树训练这些模型加剧了两个问题：1）结构优化歧义，即尽管黄金分析树有特定偏好，仍任意在结构歧义的最优语法中选择其一；2）结构简单性偏置，导致模型未充分利用规则来构建分析树。这些挑战使无监督神经语法归纳（UNGI）面临不可避免的预测错误、高方差，以及需要广泛的语法才能实现准确预测。本文解决了这些问题，并对其起源进行了全面分析。作为解决方案，我们引入了逐句分析聚焦，利用同一数据集上预训练解析器的结构偏置，来减少每个句子用于损失评估的分析池。在无监督解析基准测试中，我们的方法显著提升了性能，同时有效降低了方差和对过于简单的分析树的偏置。我们的研究促进了学习更紧凑、准确和一致的显式语法，从而提升可解释性。
英文摘要：
Neural parameterization has significantly advanced unsupervised grammar induction. However, training these models with a traditional likelihood loss for all possible parses exacerbates two issues: 1) $\textit{structural optimization ambiguity}$ that arbitrarily selects one among structurally ambiguous optimal grammars despite the specific preference of gold parses, and 2) $\textit{structural simplicity bias}$ that leads a model to underutilize rules to compose parse trees. These challenges subject unsupervised neural grammar induction (UNGI) to inevitable prediction errors, high variance, and the necessity for extensive grammars to achieve accurate predictions. This paper tackles these issues, offering a comprehensive analysis of their origins. As a solution, we introduce $\textit{sentence-wise parse-focusing}$ to reduce the parse pool per sentence for loss evaluation, using the structural bias from pre-trained parsers on the same dataset. In unsupervised parsing benchmark tests, our method significantly improves performance while effectively reducing variance and bias toward overly simplistic parses. Our research promotes learning more compact, accurate, and consistent explicit grammars, facilitating better interpretability.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
