arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2404.00899cs.CL
收藏

TM-TREK 在 SemEval-2024 任务 8:面向人机混合文本的基于大语言模型的自动边界检测

TM-TREK at SemEval-2024 Task 8: Towards LLM-Based Automatic Boundary Detection for Human-Machine Mixed Text

Samsung R&D Institute China-Beijing(三星中国北京研发院)

Xiaoyan Qu, Xiangfeng Meng

AI总结:

本文提出将人机混合文本边界检测转化为 token 分类问题,并利用 LLM 集成模型在 SemEval'24 任务 8 子任务中取得第一名,同时分析了额外层、分割损失和预训练对边界检测能力的影响。

摘要:

随着由大语言模型(LLMs)生成的文本日益普及,人们越来越关注区分 LLM 生成文本与人类撰写文本,以防止 LLM 被滥用,例如传播误导性信息和学术不端。以往研究主要集中于将文本分类为完全由人类撰写或完全由 LLM 生成,忽视了同时包含两类内容的混合文本检测。本文探索 LLM 识别人工撰写与机器生成混合文本边界的能力。我们将该任务转化为一个 token 分类问题,并将标签转折点视为边界。值得注意的是,我们的 LLM 集成模型在 SemEval'24 竞赛任务 8 的“人机混合文本检测”子任务中获得了第一名。此外,我们研究了影响 LLM 检测混合文本边界能力的因素,包括在 LLM 之上引入额外层、结合分割损失以及预训练的影响。我们的发现旨在为该领域的未来研究提供有价值的见解。

英文摘要:

With the increasing prevalence of text generated by large language models (LLMs), there is a growing concern about distinguishing between LLM-generated and human-written texts in order to prevent the misuse of LLMs, such as the dissemination of misleading information and academic dishonesty. Previous research has primarily focused on classifying text as either entirely human-written or LLM-generated, neglecting the detection of mixed texts that contain both types of content. This paper explores LLMs' ability to identify boundaries in human-written and machine-generated mixed texts. We approach this task by transforming it into a token classification problem and regard the label turning point as the boundary. Notably, our ensemble model of LLMs achieved first place in the 'Human-Machine Mixed Text Detection' sub-task of the SemEval'24 Competition Task 8. Additionally, we investigate factors that influence the capability of LLMs in detecting boundaries within mixed texts, including the incorporation of extra layers on top of LLMs, combination of segmentation loss, and the impact of pretraining. Our findings aim to provide valuable insights for future research in this area.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑