arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2601.07974cs.CL
收藏

通过语言分析解释AI生成文本检测器的泛化行为

Explaining Generalization of AI-Generated Text Detectors Through Linguistic Analysis

University of Vienna(维也纳大学);Saarland University(萨尔大学)

Yuxi Xia, Kinga Stańczak, Benjamin Roth

AI总结:

本文通过语言分析研究AI生成文本检测器的泛化行为,发现其性能与动词时态和代词频率等语言特征密切相关。

摘要:

AI生成文本检测器在领域内基准测试中能取得高准确率,但在不同生成条件(如未见过的提示、模型家族或领域)下往往难以泛化。尽管先前工作已报告了这些泛化差距,但对根本原因的见解有限。在本工作中,我们提出了一项系统研究,旨在通过语言分析来解释泛化行为。我们构建了一个涵盖6种提示策略、7种大语言模型(LLMs)和4个领域数据集的全面基准,产生了一组多样化的人工和AI生成文本。利用此数据集,我们对不同生成设置进行微调,并评估其跨提示、跨模型和跨领域泛化能力。为了解释性能差异,我们计算了80种语言特征在训练和测试条件下的特征偏移与泛化准确率之间的相关性。我们的分析发现,特定检测器和评估条件的泛化性能与动词时态使用和代词频率等语言特征显著相关。

英文摘要:

AI-text detectors achieve high accuracy on in-domain benchmarks, but often struggle to generalize across different generation conditions such as unseen prompts, model families, or domains. While prior work has reported these generalization gaps, there are limited insights about the underlying causes. In this work, we present a systematic study aimed at explaining generalization behavior through linguistic analysis. We construct a comprehensive benchmark that spans 6 prompting strategies, 7 large language models (LLMs), and 4 domain datasets, resulting in a diverse set of human- and AI-generated texts. Using this dataset, we fine-tune classification-based detectors on various generation settings and evaluate their cross-prompt, cross-model, and cross-dataset generalization. To explain the performance variance, we compute correlations between generalization accuracies and feature shifts of 80 linguistic features between training and test conditions. Our analysis reveals that generalization performance for specific detectors and evaluation conditions is significantly associated with linguistic features such as tense usage and pronoun frequency.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑