arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2407.07225cs.LGcs.CL
收藏

ConvNLP: 基于图像的AI文本检测

ConvNLP: Image-based AI Text Detection

New York University(纽约大学);IBM Research(IBM研究院)

Suriya Prakash Jambunathan, Ashwath Shankarnarayan, Parijat Dube

AI总结:

针对现有LLM生成文本检测方案计算量大、泛化性弱的问题,本文提出基于词嵌入可视化的ConvNLP检测方法,结合ZigZag ResNet与ZigZag Scheduler,实现了轻量高效且泛化性更优的AI文本检测,助力维护学术诚信。

摘要:

大语言模型(LLM)等生成式AI技术虽有革新教育领域的潜力,但其滥用引发的伦理问题削弱了这一价值,还加剧了学术不端问题。GPT-4、Llama 2等LLM在生成复杂内容、解答问题方面能力日益增强,涵盖撰写学术论文、求解复杂数学题等场景。学生正依赖这些LLM完成作业,进而损害学术诚信。现有的LLM生成文本检测方案计算量大,且往往缺乏泛化能力。本文提出一种利用词嵌入的可视化表示检测LLM生成AI文本的新方法。我们构建了一种名为ZigZag ResNet的新型卷积神经网络,还设计了用于提升泛化能力的调度器——ZigZag Scheduler。通过使用由6种不同的前沿LLM生成的文本构成的数据集开展广泛评估,我们的模型展现出强大的域内和跨域泛化能力。我们的最优模型检测AI生成文本的平均检测率(涵盖跨域和域内测试数据)达到88.35%,表现出色。通过全面的消融实验证实,相较于普通ResNet,我们的ZigZag ResNet和ZigZag Scheduler带来了近4%的性能提升。该模型的端到端推理延迟低于每句2.5毫秒。我们的方案为现有AI生成文本检测工具提供了一种轻量、计算高效且速度更快的替代方案,且具备更优的泛化性能,可帮助学术机构打击学术场景下滥用LLM的行为。通过这项工作,我们旨在为维护学术诚信原则、保障先进LLM时代学生作业的可信度贡献力量。

英文摘要:

The potentials of Generative-AI technologies like Large Language models (LLMs) to revolutionize education are undermined by ethical considerations around their misuse which worsens the problem of academic dishonesty. LLMs like GPT-4 and Llama 2 are becoming increasingly powerful in generating sophisticated content and answering questions, from writing academic essays to solving complex math problems. Students are relying on these LLMs to complete their assignments and thus compromising academic integrity. Solutions to detect LLM-generated text are compute-intensive and often lack generalization. This paper presents a novel approach for detecting LLM-generated AI-text using a visual representation of word embedding. We have formulated a novel Convolutional Neural Network called ZigZag ResNet, as well as a scheduler for improving generalization, named ZigZag Scheduler. Through extensive evaluation using datasets of text generated by six different state-of-the-art LLMs, our model demonstrates strong intra-domain and inter-domain generalization capabilities. Our best model detects AI-generated text with an impressive average detection rate (over inter- and intra-domain test data) of 88.35%. Through an exhaustive ablation study, our ZigZag ResNet and ZigZag Scheduler provide a performance improvement of nearly 4% over the vanilla ResNet. The end-to-end inference latency of our model is below 2.5ms per sentence. Our solution offers a lightweight, computationally efficient, and faster alternative to existing tools for AI-generated text detection, with better generalization performance. It can help academic institutions in their fight against the misuse of LLMs in academic settings. Through this work, we aim to contribute to safeguarding the principles of academic integrity and ensuring the trustworthiness of student work in the era of advanced LLMs.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑