重新评估无监督语法归纳中多模态信号的必要性
Re-evaluating the Need for Multimodal Signals in Unsupervised Grammar Induction
AI总结：
针对此前多模态语法归纳的提升源于弱纯文本基线的问题，提出融合大语言模型嵌入的纯文本LC-PCFG模型，在四个基准上性能优于最优多模态方法且效率更高，表明多模态输入对语法归纳或非必需。
摘要：
多模态输入对于语法归纳是必要的吗？近期研究表明，多模态训练输入能够提升语法归纳的效果。但这些提升是基于与仅使用文本的弱基线模型的比较，而这类基线模型仅在相对较少的文本数据上训练。为了确定在拥有大量文本训练数据的场景下是否仍需要多模态输入，我们设计了一个更强的纯文本基线模型，将其命名为LC-PCFG。LC-PCFG是一种C-PFCG（概率上下文无关文法），融合了纯文本大语言模型（LLM）的嵌入向量。我们采用固定的文法族，直接将LC-PCFG与多种多模态语法归纳方法进行对比，并在四个基准数据集上比较性能。与当前最优的多模态语法归纳方法相比，LC-PCFG的Corpus-F1指标相对提升最高达17%。此外，LC-PCFG的计算效率更高，与多模态方法相比，参数量最多减少85%，训练时间缩短8.8倍。这些结果表明，多模态输入对于语法归纳可能并非必要，同时凸显了强大的无视觉基线模型在评估多模态方法增益时的重要性。
英文摘要：
Are multimodal inputs necessary for grammar induction? Recent work has shown that multimodal training inputs can improve grammar induction. However, these improvements are based on comparisons to weak text-only baselines that were trained on relatively little textual data. To determine whether multimodal inputs are needed in regimes with large amounts of textual training data, we design a stronger text-only baseline, which we refer to as LC-PCFG. LC-PCFG is a C-PFCG that incorporates em-beddings from text-only large language models (LLMs). We use a fixed grammar family to directly compare LC-PCFG to various multi-modal grammar induction methods. We compare performance on four benchmark datasets. LC-PCFG provides an up to 17% relative improvement in Corpus-F1 compared to state-of-the-art multimodal grammar induction methods. LC-PCFG is also more computationally efficient, providing an up to 85% reduction in parameter count and 8.8x reduction in training time compared to multimodal approaches. These results suggest that multimodal inputs may not be necessary for grammar induction, and emphasize the importance of strong vision-free baselines for evaluating the benefit of multimodal approaches.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
