arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2408.16756cs.CL
收藏

LLM 处理粤语的能力如何?大型语言模型的粤语能力基准测试

How Well Do LLMs Handle Cantonese? Benchmarking Cantonese Capabilities of Large Language Models

The Chinese University of Hong Kong(香港中文大学);The University of Hong Kong(香港大学)

Jiyue Jiang, Pengan Chen, Liheng Chen, Sheng Wang, Qinghang Bao, Lingpeng Kong, Yu Li, Chuan Wu

AI总结:

针对粤语在 NLP 中资源与评估不足的问题,本文构建涵盖事实生成、数学逻辑、复杂推理和常识的粤语 LLM 基准,并提出研究方向与模型建议。

摘要:

大型语言模型(LLM)的快速发展改变了自然语言处理(NLP)的竞争格局,尤其是在英语和其他数据丰富的语言方面。然而,粤语等代表性不足的语言面临显著的发展差距;全球有超过 8500 万人使用粤语,考虑到粤港澳大湾区的经济重要性,以及新加坡和北美等地大量使用粤语的人群,这一问题尤其令人担忧。尽管粤语使用广泛,但它在 NLP 研究中的代表性严重不足,特别是与来自类似发达地区的其他语言相比。为弥合这些差距,我们概述了当前的粤语 NLP 方法,并引入新的基准,用于评估 LLM 在粤语事实生成、数学逻辑、复杂推理和常识方面的表现,旨在推动开源粤语 LLM 技术发展。我们还提出了未来研究方向和推荐模型,以促进粤语 LLM 的开发。

英文摘要:

The rapid evolution of large language models (LLMs) has transformed the competitive landscape in natural language processing (NLP), particularly for English and other data-rich languages. However, underrepresented languages like Cantonese, spoken by over 85 million people, face significant development gaps, which is particularly concerning given the economic significance of the Guangdong-Hong Kong-Macau Greater Bay Area, and in substantial Cantonese-speaking populations in places like Singapore and North America. Despite its wide use, Cantonese has scant representation in NLP research, especially compared to other languages from similarly developed regions. To bridge these gaps, we outline current Cantonese NLP methods and introduce new benchmarks designed to evaluate LLM performance in factual generation, mathematical logic, complex reasoning, and general knowledge in Cantonese, which aim to advance open-source Cantonese LLM technology. We also propose future research directions and recommended models to enhance Cantonese LLM development.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑