风格作为混淆因素:AI检测非母语学术写作中的假阳性
Style as a Confound: False Positives in AI Detection of Non-Native Academic Writing
- Chung-Ang University(中央大学)
- Wordvice(沃德斯(英文服务公司))
机构由 AI 辅助整理,请以论文原文为准。
AI总结:
该研究针对AI文本检测器检测非母语学术写作时的假阳性问题,利用135389份经专业编辑的文档对,发现编辑风格是关键混淆变量,不同检测器的响应存在差异,影响其公平性与可靠性。
AI中文摘要:
AI文本检测器正越来越多地应用于学术场景,但目前尚不清楚其输出结果反映的是AI作者身份本身,还是与流畅学术英语相关的更广泛语言特征。此前研究已报告非母语英语写作的假阳性率(FPR)较高,但群体层面的比较将作者身份与主题、领域和写作风格的差异相混淆。专业编辑为研究该问题提供了有用场景,因为它会改变稿件的语言形式,同时保留作者身份和内容。我们研究了某专业英语编辑服务(2018-2025年)的135389份文档对,包含非母语稿件及其经母语编辑后的版本,以评估在控制内容和作者身份的情况下,编辑如何影响检测器的响应。对于13种AI文本检测器,人类撰写文本的假阳性率差异极大,从0.0%到100.0%不等。不同检测器的响应存在差异:相同的编辑操作会使部分检测器的AI评分升高,而使另一些检测器的AI评分降低。值得注意的是,评分变化与编辑程度相关。研究结果确定专业编辑风格是AI检测器输出中的关键混淆变量,而非实现文本来源与语言风格的完全分离,这引发了对学术场景中公平性和可靠性的担忧。
英文摘要:
AI text detectors are increasingly employed in academic settings, but it remains unclear whether their outputs reflect AI authorship itself or broader linguistic features associated with polished academic English. Previous studies have reported high false-positive rates (FPRs) for non-native English writing, but population-level comparisons confound authorship with differences in topic, domain, and writing style. Professional editing provides a useful setting for examining this issue because it changes the linguistic form of manuscripts while preserving authorship and content. We examined 135,389 document pairs from a professional English editing service (2018-2025), comprising non-native manuscripts and their native-edited versions, to assess how editing affects detector responses controlling for content and authorship. For the 13 AI text detectors, FPRs for human-written texts varied widely, from 0.0% to 100.0%. Responses varied across detectors: the same edits increased AI scores in some detectors but decreased them in others. Notably, score changes correlated with the extent of editing. The findings identify professional editing style as a key confounding variable in AI detector outputs, rather than establishing a full separation of text origin from linguistic style, raising concerns about fairness and reliability in academic settings.