arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2602.05769cs.CL
收藏

不同时区,不同语言:重新审视GPT检测器对非母语者的偏见

Different Time, Different Language: Revisiting the Bias Against Non-Native Speakers in GPT Detectors

Charles University, Faculty of Mathematics and Physics(查理大学数学与物理系);University of Oslo, Language Technology Group(奥斯陆大学语言技术组)

Adnan Al Ali, Jindřich Helcl, Jindřich Libovický

AI总结:

该研究重新审视了GPT检测器对非母语者的偏见问题,发现非母语者的文本困惑度不低,并且现代检测器无需依赖困惑度即可有效工作。

摘要:

在ChatGPT发布后,基于大语言模型(LLM)的助手得到了广泛普及。然而,由于难以区分人类写作和生成文本,人们在学术界对它们的滥用表示担忧。为应对这一问题,已开发出自动技术并显示出一定的有效性。然而,先前的研究表明,这些方法常常错误地将非母语者的论文标记为生成文本,因为这些文本从LLM中提取出的困惑度较低,这被认为是检测器的关键特征。我们两年后重新审视这些说法,特别是在捷克语环境下。我们证明,非捷克语母语者的文本的困惑度并不低于母语者。我们进一步检查了三种不同的检测器家族,发现没有系统性的对非母语者的偏见。最后,我们证明现代检测器在不依赖困惑度的情况下能够有效运行。

英文摘要:

LLM-based assistants have been widely popularised after the release of ChatGPT. Concerns have been raised about their misuse in academia, given the difficulty of distinguishing between human-written and generated text. To combat this, automated techniques have been developed and shown to be effective, to some extent. However, prior work suggests that these methods often falsely flag essays from non-native speakers as generated, due to their low perplexity extracted from an LLM, which is supposedly a key feature of the detectors. We revisit these statements two years later, specifically in the Czech language setting. We show that the perplexity of texts from non-native speakers of Czech is not lower than that of native speakers. We further examine detectors from three separate families and find no systematic bias against non-native speakers. Finally, we demonstrate that contemporary detectors operate effectively without relying on perplexity.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑