TSM-Bench：在真实维基百科编辑实践中检测LLM生成的文本
TSM-Bench: Detecting LLM-Generated Text in Real-World Wikipedia Editing Practices
AI总结：
针对维基百科等用户生成内容平台，提出多语言、多生成器、多任务的TSM-Bench基准，发现现有检测器在任务特定MGT上准确率下降10-40%，且存在泛化不对称性。
摘要：
自动检测机器生成文本（MGT）对于维护维基百科等用户生成内容（UGC）平台的知识完整性至关重要。现有的检测基准主要关注 extit{通用}文本生成任务（例如，“写一篇关于机器学习的文章。”）。然而，编辑者经常使用LLM进行特定的写作任务（例如，摘要）。这些 extit{任务特定}的MGT实例由于其受限的任务制定和上下文条件，往往更接近人类撰写的文本。在这项工作中，我们展示了一系列最先进的MGT检测器在识别反映维基百科真实编辑的任务特定MGT时存在困难。我们引入了 extsc{TSM-Bench}，这是一个多语言、多生成器和 extit{多任务}基准，用于评估MGT检测器在常见的真实维基百科编辑任务上的表现。我们的发现表明：（ extit{i}）与之前的基准相比，平均检测准确率下降了10-40%；（ extit{ii}）存在泛化不对称性：在任务特定数据上微调能够泛化到通用数据——甚至跨领域——但反之则不然。我们证明，仅在通用MGT上微调的模型会过度拟合机器生成的表面伪影。我们的结果表明，与之前的基准相比，大多数检测器在UGC平台等真实世界上下文中仍不可靠。因此， extsc{TSM-Bench}为开发和评估未来模型提供了关键基础。
英文摘要：
Automatically detecting machine-generated text (MGT) is critical to maintaining the knowledge integrity of user-generated content (UGC) platforms such as Wikipedia. Existing detection benchmarks primarily focus on \textit{generic} text generation tasks (e.g., ``Write an article about machine learning.''). However, editors frequently employ LLMs for specific writing tasks (e.g., summarisation). These \textit{task-specific} MGT instances tend to resemble human-written text more closely due to their constrained task formulation and contextual conditioning. In this work, we show that a range of SOTA MGT detectors struggle to identify task-specific MGT reflecting real-world editing on Wikipedia. We introduce \textsc{TSM-Bench}, a multilingual, multi-generator, and \textit{multi-task} benchmark for evaluating MGT detectors on common, real-world Wikipedia editing tasks. Our findings demonstrate that (\textit{i}) average detection accuracy drops by 10--40\% compared to prior benchmarks, and (\textit{ii}) a generalisation asymmetry exists: fine-tuning on task-specific data enables generalisation to generic data -- even across domains -- but not vice versa. We demonstrate that models fine-tuned exclusively on generic MGT overfit to superficial artefacts of machine generation. Our results suggest that, in contrast to prior benchmarks, most detectors remain unreliable for automated detection in real-world contexts such as UGC platforms. \textsc{TSM-Bench} therefore provides a critical foundation for developing and evaluating future models.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
