基于压缩的编辑距离评估人类对LLM生成文本的编辑努力
Assessing Human Editing Effort on LLM-Generated Texts via Compression-Based Edit Distance
AI总结：
本文提出了一种基于Lempel-Ziv-77算法的压缩编辑距离指标，用于准确量化人类对LLM生成文本的后期编辑努力，实验证明该指标与实际编辑时间高度相关且能高效捕获复杂编辑。
摘要：
评估人类对大型语言模型（LLM）生成文本的编辑程度，对于理解人机交互和提升自动文本生成系统的质量至关重要。现有的编辑距离指标，如Levenshtein、BLEU、ROUGE和TER，往往无法准确衡量后期编辑所需的努力，尤其是当编辑涉及大量修改（如块操作）时。在本文中，我们引入了一种基于压缩的新型编辑距离指标，该指标基于Lempel-Ziv-77算法，旨在量化应用于LLM生成文本的后期编辑量。我们的方法利用文本压缩的特性来衡量原始文本与编辑后文本之间的信息差异。通过对真实人类编辑数据集的实验，我们证明了我们提出的指标与实际编辑时间和努力高度相关。我们还表明，LLM表现出对编辑速度的隐式理解，这与我们的指标非常吻合。此外，我们将我们的指标与现有指标进行了比较，突出了其在以线性计算效率捕获复杂编辑方面的优势。我们的代码和数据可在以下网址获取：https://github.com/NDV-tiime/CompressionDistance
英文摘要：
Assessing the extent of human edits on texts generated by Large Language Models (LLMs) is crucial to understanding the human-AI interactions and improving the quality of automated text generation systems. Existing edit distance metrics, such as Levenshtein, BLEU, ROUGE, and TER, often fail to accurately measure the effort required for post-editing, especially when edits involve substantial modifications, such as block operations. In this paper, we introduce a novel compression-based edit distance metric grounded in the Lempel-Ziv-77 algorithm, designed to quantify the amount of post-editing applied to LLM-generated texts. Our method leverages the properties of text compression to measure the informational difference between the original and edited texts. Through experiments on real-world human edits datasets, we demonstrate that our proposed metric is highly correlated with actual edit time and effort. We also show that LLMs exhibit an implicit understanding of editing speed, that aligns well with our metric. Furthermore, we compare our metric with existing ones, highlighting its advantages in capturing complex edits with linear computational efficiency. Our code and data are available at: https://github.com/NDV-tiime/CompressionDistance
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
