评估风格个性化文本生成：挑战与方向
Evaluating Style-Personalized Text Generation: Challenges and Directions
AI总结：
研究风格个性化文本生成面临的挑战，通过提出的风格判别基准评估常用指标，发现多样评估指标集成效果更佳，为可靠评估该文本生成提供指导。
摘要：
随着大语言模型的兴起及其定制输出能力，风格个性化文本生成成为快速发展的研究领域。但风格个性化具有独特性与挑战性，现有基准和指标存在局限。本文批判性审视常用指标如BLEU、嵌入和基于大语言模型的评判等。利用提出的风格判别基准在三种评估设置下评估这些指标，发现多样评估指标的集成优于单一评估方法，并给出可靠评估风格个性化文本生成的指导。
英文摘要：
With the surge of large language models (LLMs) and their ability to produce customized output, style-personalized text generation--"write like me"--has become a rapidly growing area of interest. However, style personalization is highly specific, relative to every user, and depends strongly on the pragmatic context, which makes it uniquely challenging. Although prior research has introduced benchmarks and metrics for this area, they tend to be non-standardized and have known limitations (e.g., poor correlation with human subjects). LLMs have been found to not capture author-specific style well, it follows that the metrics themselves must be scrutinized carefully. In this work we critically examine the effectiveness of the most common metrics used in the field, such as BLEU, embeddings, and LLMs-as-judges. We evaluate these metrics using our proposed style discrimination benchmark, which spans eight diverse writing tasks across three evaluation settings: domain discrimination, authorship attribution, and LLM-generated personalized vs non-personalized discrimination. We find strong evidence that employing ensembles of diverse evaluation metrics consistently outperforms single-evaluator methods, and conclude by providing guidance on how to reliably assess style-personalized text generation.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
