PersonalSum：面向大语言模型的用户主观引导型个性化摘要数据集
PersonalSum: A User-Subjective Guided Personalized Summarization Dataset for Large Language Models
AI总结：
针对现有个性化摘要研究缺乏普通大众主动标注数据集的问题，本文提出首个探究读者关注与LLM通用摘要差异的人工标注个性化摘要数据集PersonalSum，并验证了实体/主题等信号对个性化摘要生成的影响，指出该任务对现有LLM仍具挑战。
摘要：
近年来，随着自然语言处理技术的快速发展，大量研究表明，根据人类评估结果，大语言模型（LLM）生成的通用摘要有时甚至能超越记者等专业人士标注的摘要。然而，关于这些通用摘要能否满足普通用户个性化需求的研究仍十分有限，最大的阻碍是缺乏来自普通大众的人工标注数据集。现有的个性化摘要研究往往依赖于从通用摘要数据集构建的伪数据集，或是聚焦于特定命名实体或其他维度（如生成摘要的长度和具体程度）的可控任务，这些数据均来自假设性任务，并非标注者主动生成。为填补这一空白，我们提出了一个高质量、人工标注的个性化生成式摘要数据集PersonalSum。该数据集是首个探究普通读者关注重点与LLM生成的通用摘要是否存在差异的数据集，包含用户画像、附带原文源句的个性化摘要，以及机器生成的通用摘要及其对应源文本。我们在少样本上下文学习场景下，探究了可能影响LLM生成个性化摘要的若干个人信号——实体/主题、文章情节与文章结构。我们的初步结果与分析表明，实体/主题只是影响用户多样化偏好的关键因素之一，个性化摘要对现有LLM而言仍是一项重大挑战。
英文摘要：
With the rapid advancement of Natural Language Processing in recent years, numerous studies have shown that generic summaries generated by Large Language Models (LLMs) can sometimes surpass those annotated by experts, such as journalists, according to human evaluations. However, there is limited research on whether these generic summaries meet the individual needs of ordinary people. The biggest obstacle is the lack of human-annotated datasets from the general public. Existing work on personalized summarization often relies on pseudo datasets created from generic summarization datasets or controllable tasks that focus on specific named entities or other aspects, such as the length and specificity of generated summaries, collected from hypothetical tasks without the annotators' initiative. To bridge this gap, we propose a high-quality, personalized, manually annotated abstractive summarization dataset called PersonalSum. This dataset is the first to investigate whether the focus of public readers differs from the generic summaries generated by LLMs. It includes user profiles, personalized summaries accompanied by source sentences from given articles, and machine-generated generic summaries along with their sources. We investigate several personal signals - entities/topics, plot, and structure of articles - that may affect the generation of personalized summaries using LLMs in a few-shot in-context learning scenario. Our preliminary results and analysis indicate that entities/topics are merely one of the key factors that impact the diverse preferences of users, and personalized summarization remains a significant challenge for existing LLMs.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
