arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2411.04025cs.CL
收藏

使用GPT进行提示工程以实现低资源德拉维达语言词级语码混合语言识别

Prompt Engineering Using GPT for Word-Level Code-Mixed Language Identification in Low-Resource Dravidian Languages

IIT Kharagpur(印度理工学院克勒格布尔分校)

Aniket Deroy, Subhankar Maity

AI总结:

本文利用GPT-3.5 Turbo提示工程方法解决低资源德拉维达语言的词级语码混合语言识别问题,实验表明卡纳达语模型表现优于泰米尔语模型。

摘要:

语言识别(LI)对于各类自然语言处理任务至关重要,是情感分析、机器翻译和信息检索等应用的基础步骤。在印度等多语言社会中,尤其是活跃于社交媒体的年轻人中,文本常常呈现语码混合现象,在不同语言层级上将本地语言与英语混合使用。这一现象给语言识别系统带来了严峻挑战,尤其是当多种语言在单个单词内部交织时。德拉维达语言在印度南部广泛使用,具有丰富的形态结构,但在数字平台上的代表性不足,导致人们采用罗马字母或混合文字进行交流。本文提出了一种基于提示的方法,用于一项旨在解决德拉维达语言词级语言识别挑战的共享任务。在这项工作中,我们利用GPT-3.5 Turbo来探究大语言模型是否能够将单词正确分类到正确的类别中。我们的研究结果表明,卡纳达语模型在大多数指标上持续优于泰米尔语模型,表明其在识别和分类卡纳达语实例方面具有更高的准确性和可靠性。相比之下,泰米尔语模型表现中等,尤其在精确率和召回率方面需要改进。

英文摘要:

Language Identification (LI) is crucial for various natural language processing tasks, serving as a foundational step in applications such as sentiment analysis, machine translation, and information retrieval. In multilingual societies like India, particularly among the youth engaging on social media, text often exhibits code-mixing, blending local languages with English at different linguistic levels. This phenomenon presents formidable challenges for LI systems, especially when languages intermingle within single words. Dravidian languages, prevalent in southern India, possess rich morphological structures yet suffer from under-representation in digital platforms, leading to the adoption of Roman or hybrid scripts for communication. This paper introduces a prompt based method for a shared task aimed at addressing word-level LI challenges in Dravidian languages. In this work, we leveraged GPT-3.5 Turbo to understand whether the large language models is able to correctly classify words into correct categories. Our findings show that the Kannada model consistently outperformed the Tamil model across most metrics, indicating a higher accuracy and reliability in identifying and categorizing Kannada language instances. In contrast, the Tamil model showed moderate performance, particularly needing improvement in precision and recall.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑