用预训练的语言模型欺骗MOSS检测
Fooling MOSS Detection with Pretrained Language Models
摘要：
随着人工智能（AI）技术在社会中变得越来越强大和突出，它们的滥用越来越令人担忧。在教育环境中，人工智能技术可以被学生用来在作业和考试中作弊。在本文中，我们将探讨Transformers是否可以用于解决入门级编程任务，同时绕过常用的AI工具来检测软件之间的相似性。我们发现，使用GPT-J的学生[Wang and Komatsuzaki，2021]可以完成入门级的编程作业，而不会引起MOSS [Aiken，2000]的怀疑，MOSS是一种广泛使用的软件相似性和剽窃检测工具。尽管GPT-J没有接受过有关问题的培训，也没有提供任何可供借鉴的例子，但情况仍然如此。我们进一步发现，GPT-J编写的代码在结构上是多样的，缺乏任何特定的告诉，未来的剽窃检测技术可能会用来尝试识别算法生成的代码。最后，我们讨论的伦理和教育的影响，大语言模型和未来的研究方向。
英文摘要：
As artificial intelligence (AI) technologies become increasingly powerful and prominent in society, their misuse is a growing concern. In educational settings, AI technologies could be used by students to cheat on assignments and exams. In this paper we explore whether transformers can be used to solve introductory level programming assignments while bypassing commonly used AI tools to detect similarities between pieces of software. We find that a student using GPT-J [Wang and Komatsuzaki, 2021] can complete introductory level programming assignments without triggering suspicion from MOSS [Aiken, 2000], a widely used software similarity and plagiarism detection tool. This holds despite the fact that GPT-J was not trained on the problems in question and is not provided with any examples to work from. We further find that the code written by GPT-J is diverse in structure, lacking any particular tells that future plagiarism detection techniques may use to try to identify algorithmically generated code. We conclude with a discussion of the ethical and educational implications of large language models and directions for future research.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
