arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2404.16653cs.CLcs.AI
收藏

大规模语言模型(LLMs)中的语言歧义分析

Análise de ambiguidade linguística em modelos de linguagem de grande escala (LLMs)

Attenty Sistemas de Software(Attenty软件系统公司);Universidade Federal de Sergipe(塞尔希培联邦大学)
Lavínia de Carvalho Moraes, Irene Cristina Silvério, Rafael Alexandre Sousa Marques, Bianca de Castro Anaia, Dandara Freitas de Paula, Maria Carolina Schincario… 展开作者

Lavínia de Carvalho Moraes, Irene Cristina Silvério, Rafael Alexandre Sousa Marques, Bianca de Castro Anaia, Dandara Freitas de Paula, Maria Carolina Schincariol de Faria, Iury Cleveston, Alana de Santana Correia, Raquel Meister Ko Freitag

AI总结:

本研究分析了ChatGPT和Gemini等大规模语言模型在巴西葡萄牙语语义、句法和词汇歧义处理上的表现,通过120个句子的语料库进行分类、解释和消歧测试,发现模型响应存在错误且解释不一致,最高准确率仅49.58%,揭示了监督学习描述性研究的必要性。

摘要:

尽管Transformers和BERT等架构取得了进展,但语言歧义仍然是自然语言处理(NLP)系统面临的重大挑战。受ChatGPT和Gemini(在2023年,该人工智能被称为Bard)等指令模型近期成功的启发,本研究旨在分析和讨论这些模型中的语言歧义,重点关注巴西葡萄牙语中普遍存在的三种歧义:语义、句法和词汇歧义。我们创建了一个包含120个句子的语料库,包含歧义和非歧义句子,用于分类、解释和消歧。研究还通过为每种歧义类型征集句子集,探索了模型生成歧义句子的能力。结果基于公认的语言学参考资料进行了定性分析,并根据所获响应的准确性进行了定量评估。结果表明,即使是ChatGPT和Gemini等最先进的模型,其响应也存在错误和缺陷,解释往往不一致。此外,准确率最高仅为49.58%,表明需要对监督学习进行描述性研究。

英文摘要:

Linguistic ambiguity continues to represent a significant challenge for natural language processing (NLP) systems, notwithstanding the advancements in architectures such as Transformers and BERT. Inspired by the recent success of instructional models like ChatGPT and Gemini (In 2023, the artificial intelligence was called Bard.), this study aims to analyze and discuss linguistic ambiguity within these models, focusing on three types prevalent in Brazilian Portuguese: semantic, syntactic, and lexical ambiguity. We create a corpus comprising 120 sentences, both ambiguous and unambiguous, for classification, explanation, and disambiguation. The models capability to generate ambiguous sentences was also explored by soliciting sets of sentences for each type of ambiguity. The results underwent qualitative analysis, drawing on recognized linguistic references, and quantitative assessment based on the accuracy of the responses obtained. It was evidenced that even the most sophisticated models, such as ChatGPT and Gemini, exhibit errors and deficiencies in their responses, with explanations often providing inconsistent. Furthermore, the accuracy peaked at 49.58 percent, indicating the need for descriptive studies for supervised learning.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑