在多语言和语码转换语境下评估大型语言模型对多词表达的处理能力
Evaluating Large Language Models on Multiword Expressions in Multilingual and Code-Switched Contexts
AI总结：
本研究评估最先进语言模型在多语言和语码转换语境下处理潜在习语化多词表达歧义的能力，发现包括GPT-4在内的最新模型在检测和语义任务中均未能超越xlm-roBERTa-base基线，表明多词表达仍是模型的重大挑战。
摘要：
多词表达以非组合性含义和句法不规则性为特征，是微妙语言的典型代表。这些表达既可字面使用也可习语化使用，从而导致含义的显著变化。尽管大型语言模型在许多任务中表现出色，但它们处理此类语言微妙之处的能力仍不确定。因此，本研究评估了最先进的语言模型如何处理潜在习语化多词表达的歧义，特别是在较低频语境中，模型不太可能依赖记忆。通过在英语之外评估葡萄牙语和加利西亚语模型，并使用新颖的语码转换数据集和新颖任务，我们发现大型语言模型尽管有其优势，但在处理微妙语言时仍显吃力。具体而言，包括GPT-4在内的最新模型在检测和语义任务中均未能超越xlm-roBERTa-base基线，在我们引入的新任务上表现尤差，尽管该任务与现有任务相似。总体而言，结果表明多词表达，尤其是歧义表达，对模型仍是挑战。
英文摘要：
Multiword expressions, characterised by non-compositional meanings and syntactic irregularities, are an example of nuanced language. These expressions can be used literally or idiomatically, leading to significant changes in meaning. While large language models have demonstrated strong performance across many tasks, their ability to handle such linguistic subtleties remains uncertain. Therefore, this study evaluates how state-of-the-art language models process the ambiguity of potentially idiomatic multiword expressions, particularly in contexts that are less frequent, where models are less likely to rely on memorisation. By evaluating models across in Portuguese and Galician, in addition to English, and using a novel code-switched dataset and a novel task, we find that large language models, despite their strengths, struggle with nuanced language. In particular, we find that the latest models, including GPT-4, fail to outperform the xlm-roBERTa-base baselines in both detection and semantic tasks, with especially poor performance on the novel tasks we introduce, despite its similarity to existing tasks. Overall, our results demonstrate that multiword expressions, especially those which are ambiguous, continue to be a challenge to models.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
