面向教育中人类与AI协作混合论文的自动边界检测
Towards Automatic Boundary Detection for Human-AI Collaborative Hybrid Essay in Education
AI总结：
本研究针对人类与AI协作撰写的混合文本，提出了一种基于原型距离的两步边界检测方法，在域内和域外评估中分别提升22%和18%。
摘要：
近期的大型语言模型（LLMs），例如ChatGPT，在提供特定指令时能够生成类似人类且流畅的回应。虽然承认技术进步带来的便利，教育工作者也担心学生可能利用LLMs完成写作作业并将其冒充为原创作品。尽管由于这些担忧已经进行了许多AI内容检测研究，但大多数先前的研究将AI内容检测建模为分类问题，假设文本要么完全由人类撰写，要么完全由AI生成。在本研究中，我们探讨了AI内容检测在一个鲜有探索但现实的环境中的应用，即待检测文本由人类和生成式LLMs协作撰写（即混合文本）。我们首先将检测任务形式化为从给定混合文本中识别人类撰写内容与AI生成内容之间的转换点（边界检测）。然后，我们提出了一种两步方法，其中（1）在编码器训练过程中将AI生成的内容与人类撰写的内容分离；（2）计算每两个相邻原型之间的距离，并假设边界存在于距离彼此最远的两个相邻原型之间。通过大量实验，我们观察到以下主要发现：（1）所提出的方法在不同实验设置下始终优于基线方法；（2）编码器训练过程可以显著提升所提出方法的性能；（3）在检测单边界混合论文的边界时，所提出的方法可以通过采用相对较大的原型大小来增强，导致在域内评估中提升22%，在域外评估中提升18%。
英文摘要：
The recent large language models (LLMs), e.g., ChatGPT, have been able to generate human-like and fluent responses when provided with specific instructions. While admitting the convenience brought by technological advancement, educators also have concerns that students might leverage LLMs to complete their writing assignments and pass them off as their original work. Although many AI content detection studies have been conducted as a result of such concerns, most of these prior studies modeled AI content detection as a classification problem, assuming that a text is either entirely human-written or entirely AI-generated. In this study, we investigated AI content detection in a rarely explored yet realistic setting where the text to be detected is collaboratively written by human and generative LLMs (i.e., hybrid text). We first formalized the detection task as identifying the transition points between human-written content and AI-generated content from a given hybrid text (boundary detection). Then we proposed a two-step approach where we (1) separated AI-generated content from human-written content during the encoder training process; and (2) calculated the distances between every two adjacent prototypes and assumed that the boundaries exist between the two adjacent prototypes that have the furthest distance from each other. Through extensive experiments, we observed the following main findings: (1) the proposed approach consistently outperformed the baseline methods across different experiment settings; (2) the encoder training process can significantly boost the performance of the proposed approach; (3) when detecting boundaries for single-boundary hybrid essays, the proposed approach could be enhanced by adopting a relatively large prototype size, leading to a 22% improvement in the In-Domain evaluation and an 18% improvement in the Out-of-Domain evaluation.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
