大语言模型作为多源观点摘要的架构师与评论家
LLMs as Architects and Critics for Multi-Source Opinion Summarization
AI总结：
针对多源观点摘要（M-OS）研究空白与评估数据集缺失问题，提出M-OS-EVAL基准数据集，探索LLMs在该任务中的应用，实验显示其提升用户参与度且与人类判断对齐性优于先前方法。
摘要：
多源观点摘要（M-OS）在传统观点摘要基础上扩展，纳入产品描述、关键特性、规格、评分等元数据及评论等额外来源。这种整合生成的综合摘要既捕捉主观观点，又涵盖客观产品属性，对明智决策至关重要。尽管大语言模型（LLMs）在多种自然语言处理（NLP）任务中表现出色，但它们在M-OS中的潜力仍未被充分探索。此外，该任务缺乏评估数据集阻碍了进一步发展。为填补这一空白，我们引入M-OS-EVAL基准数据集，从流畅性、连贯性、相关性、忠实性、方面覆盖度、情感一致性、特异性7个关键维度评估多源观点摘要。用户研究显示，M-OS显著提升用户参与度，平均87%的参与者更偏好M-OS而非传统观点摘要；事实增强的摘要提升用户参与度，且M-OS-PROMPTS与人类判断的对齐性更强，平均斯皮尔曼相关系数ρ=0.74，超过先前方法的性能。
英文摘要：
Multi-source Opinion Summarization (M-OS) extends beyond traditional opinion summarization by incorporating additional sources of product metadata such as descriptions, key features, specifications, and ratings, alongside reviews. This integration results in comprehensive summaries that capture both subjective opinions and objective product attributes essential for informed decision-making. While Large Language Models (LLMs) have shown significant success in various Natural Language Processing (NLP) tasks, their potential in M-OS remains largely unexplored. Additionally, the lack of evaluation datasets for this task has impeded further advancements. To bridge this gap, we introduce M-OS-EVAL, a benchmark dataset for evaluating multi-source opinion summaries across 7 key dimensions: fluency, coherence, relevance, faithfulness, aspect coverage, sentiment consistency, specificity. Our results demonstrate that M-OS significantly enhances user engagement, as evidenced by a user study in which, on average, 87% of participants preferred M-OS over opinion summaries. Our experiments demonstrate that factually enriched summaries enhance user engagement. Notably, M-OS-PROMPTS exhibit stronger alignment with human judgment, achieving an average Spearman correlation of \r{ho} = 0.74, which surpasses the performance of previous methodologies.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
