ExPerT：个性化长文本生成的有效且可解释评估
ExPerT: Effective and Explainable Evaluation of Personalized Long-Form Text Generation
AI总结：
本文提出可解释的个性化文本生成评估框架ExPerT，通过LLM提取并匹配原子方面，从内容与写作风格评估生成文本与参考文本的一致性，并生成细粒度解释，在人类判断一致性上相对提升7.2%。
摘要：
评估由大型语言模型（LLMs）生成的个性化文本具有挑战性，因为只有LLM用户（即提示作者）能够可靠地评估输出，但在不同研究中重新联系同一批个体是不可行的。本文通过引入ExPerT（一个可解释的基于参考的评估框架）来应对个性化文本生成评估的挑战。ExPerT利用LLM从生成文本和参考文本中提取原子方面及其证据，匹配这些方面，并基于内容和写作风格——个性化文本生成中的两个关键属性——评估它们的一致性。此外，ExPerT为评估过程的每一步生成详细、细粒度的解释，增强了透明度和可解释性。我们的实验表明，与最先进的文本生成评估方法相比，ExPerT在与人类判断的一致性上实现了7.2%的相对提升。此外，人类评估者将ExPerT解释的可用性评为4.7分（满分5分），突显了其在使评估决策更具可解释性方面的有效性。
英文摘要：
Evaluating personalized text generated by large language models (LLMs) is challenging, as only the LLM user, i.e., prompt author, can reliably assess the output, but re-engaging the same individuals across studies is infeasible. This paper addresses the challenge of evaluating personalized text generation by introducing ExPerT, an explainable reference-based evaluation framework. ExPerT leverages an LLM to extract atomic aspects and their evidence from the generated and reference texts, match the aspects, and evaluate their alignment based on content and writing style -- two key attributes in personalized text generation. Additionally, ExPerT generates detailed, fine-grained explanations for every step of the evaluation process, enhancing transparency and interpretability. Our experiments demonstrate that ExPerT achieves a 7.2% relative improvement in alignment with human judgments compared to the state-of-the-art text generation evaluation methods. Furthermore, human evaluators rated the usability of ExPerT's explanations at 4.7 out of 5, highlighting its effectiveness in making evaluation decisions more interpretable.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
