arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2312.01672cs.CLcs.AI
收藏

STADEE:基于统计的机器生成文本深度检测方法

STADEE: STAtistics-based DEEp Detection of Machine Generated Text

School of Information and Software Engineering(信息与软件工程学院);University of Electronic Science and Technology of China(电子科技大学)

Zheng Chen, Huming Liu

AI总结:

STADEE通过融合统计文本特征与深度分类器来检测机器生成文本,在域内、域外及真实场景中均优于传统统计方法和微调预训练语言模型,域内F1达87.05%。

摘要:

我们提出STADEE,一种基于统计的深度检测方法,用于识别机器生成文本,以解决当前方法严重依赖微调预训练语言模型(PLMs)的局限性。STADEE将关键统计文本特征与深度分类器相结合,重点关注token概率和累积概率等方面,这对于处理核采样至关重要。在多种数据集和场景(域内、域外和真实场景)中进行测试后,STADEE展现出优越的性能,在域内取得了87.05%的F1分数,并超越了传统统计方法和微调后的PLMs,尤其是在域外和真实场景设置中表现突出,彰显了其有效性和泛化能力。

英文摘要:

We present STADEE, a \textbf{STA}tistics-based \textbf{DEE}p detection method to identify machine-generated text, addressing the limitations of current methods that rely heavily on fine-tuning pre-trained language models (PLMs). STADEE integrates key statistical text features with a deep classifier, focusing on aspects like token probability and cumulative probability, crucial for handling nucleus sampling. Tested across diverse datasets and scenarios (in-domain, out-of-domain, and in-the-wild), STADEE demonstrates superior performance, achieving an 87.05% F1 score in-domain and outperforming both traditional statistical methods and fine-tuned PLMs, especially in out-of-domain and in-the-wild settings, highlighting its effectiveness and generalizability.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑