反图灵测试($CT^2$):面向印地语的AI生成文本检测研究——基于印地语AI可检测性指数($ADI_{hi}$)的大语言模型排名
Counter Turing Test ($CT^2$): Investigating AI-Generated Text Detection for Hindi -- Ranking LLMs based on Hindi AI Detectability Index ($ADI_{hi}$)
AI总结:
针对印地语AI生成文本检测需求,研究检验26个大语言模型的印地语生成能力,推出对应数据集,评估五种检测技术效果,并提出印地语AI可检测性指数以刻画相关演进格局。
摘要:
大语言模型(LLM)的广泛应用以及多语种LLM的普及,引发了人们对AI生成文本滥用相关潜在风险与负面影响的担忧,因此亟需提升警惕性。尽管这些模型主要针对英语训练,但它们在覆盖几乎整个互联网的海量数据集上进行的大规模训练,使其具备了在众多其他语言中良好运行的能力。AI生成文本检测(AGTD)已成为迅速受到研究关注的课题,学界已提出若干初步方法,随后很快又出现了绕过检测的技术。本文针对印度语言印地语的AGTD问题展开研究。主要贡献分为四个方面:i)检验了26个LLM生成印地语文本的能力;ii)推出了印地语AI生成新闻文章数据集$AG_{hi}$;iii)评估了ConDA、J-Guard、RADAR、RAIDAR和本征维度估计这五种最新提出的AGTD技术检测AI生成印地语文本的效果;iv)提出了印地语AI可检测性指数$ADI_{hi}$,该指数呈现出一个谱系,可用于理解AI生成印地语文本表达能力不断演进的格局。代码与数据集可在https://github.com/ishank31/Counter_Turing_Test获取。
英文摘要:
The widespread adoption of Large Language Models (LLMs) and awareness around multilingual LLMs have raised concerns regarding the potential risks and repercussions linked to the misapplication of AI-generated text, necessitating increased vigilance. While these models are primarily trained for English, their extensive training on vast datasets covering almost the entire web, equips them with capabilities to perform well in numerous other languages. AI-Generated Text Detection (AGTD) has emerged as a topic that has already received immediate attention in research, with some initial methods having been proposed, soon followed by the emergence of techniques to bypass detection. In this paper, we report our investigation on AGTD for an indic language Hindi. Our major contributions are in four folds: i) examined 26 LLMs to evaluate their proficiency in generating Hindi text, ii) introducing the AI-generated news article in Hindi ($AG_{hi}$) dataset, iii) evaluated the effectiveness of five recently proposed AGTD techniques: ConDA, J-Guard, RADAR, RAIDAR and Intrinsic Dimension Estimation for detecting AI-generated Hindi text, iv) proposed Hindi AI Detectability Index ($ADI_{hi}$) which shows a spectrum to understand the evolving landscape of eloquence of AI-generated text in Hindi. The code and dataset is available at https://github.com/ishank31/Counter_Turing_Test
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。