arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2301.01967cs.CL
收藏

语码转换研究综述:语言技术的语言学和社会学视角

A Survey of Code-switching: Linguistic and Social Perspectives for Language Technologies

Ghent University(根特大学);Microsoft Research India(微软印度研究院);UT at Austin(德克萨斯大学奥斯汀分校)

A. Seza Doğruöz, Sunayana Sitaram, Barbara E. Bullock, Almeida Jacqueline Toribio

摘要:

近年来,对代表多种语言的数据的分析在计算语言学家中得到了普及。到目前为止,大部分研究主要集中在计算方法的改进上,在很大程度上忽略了语言学长期文献中广泛讨论的C-S的语言和社会方面。为了填补这一空白,我们提供了一个调查的代码转换(C-S)涵盖在语言学文献与语言技术的关键问题的反思。从语言学的角度来看,我们提供了一个概述的结构和功能模式的C-S集中在文学从欧洲和印度的背景下,高度多语言的地区。从语言技术的角度来看,我们讨论了如何大量的语言模型未能代表不同的C-S类型,由于缺乏适当的训练数据,缺乏强大的C-S(跨多语言的情况和C-S类型)的评估基准,以及缺乏端到端的系统,涵盖社会语言学方面的C-S。我们的调查将是朝着计算科学家和语言学家在多语言和C-S方面共同利益的结果迈出的一步。

英文摘要:

The analysis of data in which multiple languages are represented has gained popularity among computational linguists in recent years. So far, much of this research focuses mainly on the improvement of computational methods and largely ignores linguistic and social aspects of C-S discussed across a wide range of languages within the long-established literature in linguistics. To fill this gap, we offer a survey of code-switching (C-S) covering the literature in linguistics with a reflection on the key issues in language technologies. From the linguistic perspective, we provide an overview of structural and functional patterns of C-S focusing on the literature from European and Indian contexts as highly multilingual areas. From the language technologies perspective, we discuss how massive language models fail to represent diverse C-S types due to lack of appropriate training data, lack of robust evaluation benchmarks for C-S (across multilingual situations and types of C-S) and lack of end-to-end systems that cover sociolinguistic aspects of C-S as well. Our survey will be a step towards an outcome of mutual benefit for computational scientists and linguists with a shared interest in multilingualism and C-S.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑