“我对自己更了解，但并非真正了解”：LLM 在检测和解释LLM生成文本方面的能力如何？
"I know myself better, but not really greatly": How Well Can LLMs Detect and Explain LLM-Generated Texts?
AI总结：
本文研究LLM检测和解释生成文本的能力，发现自我检测优于交叉检测，三元分类提升性能，但存在依赖不准确特征、幻觉和推理缺陷等局限。
摘要：
区分人类生成文本与LLM生成文本至关重要，因为LLM的滥用存在风险。本文研究了当前LLM在两种设置下的检测和解释能力：二元分类（人类 vs. LLM生成）和三元分类（包括“不确定”类别）。我们评估了6个不同规模的闭源和开源LLM，发现自我检测（LLM识别自身输出）始终优于交叉检测（识别其他LLM的输出），尽管两者均未达到最优。引入三元分类框架提高了所有模型的检测准确性和解释质量。通过使用我们人工标注的数据集进行全面的定量和定性分析，我们识别出关键的解释失败，主要依赖于不准确的特征、幻觉和有缺陷的推理。我们的研究结果强调了当前LLM在自我检测和自我解释方面的局限性，突出了进一步研究以解决过拟合和增强泛化能力的必要性。
英文摘要：
Distinguishing between human- and LLM-generated texts is crucial given the risks associated with misuse of LLMs. This paper investigates detection and explanation capabilities of current LLMs across two settings: binary (human vs. LLM-generated) and ternary classification (including an ``undecided'' class). We evaluate 6 close- and open-source LLMs of varying sizes and find that self-detection (LLMs identifying their own outputs) consistently outperforms cross-detection (identifying outputs from other LLMs), though both remain suboptimal. Introducing a ternary classification framework improves both detection accuracy and explanation quality across all models. Through comprehensive quantitative and qualitative analyses using our human-annotated dataset, we identify key explanation failures, primarily reliance on inaccurate features, hallucinations, and flawed reasoning. Our findings underscore the limitations of current LLMs in self-detection and self-explanation, highlighting the need for further research to address overfitting and enhance generalizability.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
