arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2408.11609cs.CLcs.AI
收藏

Xinyu:一个用于评论生成的高效LLM系统

Xinyu: An Efficient LLM-based System for Commentary Generation

Zhejiang University(浙江大学);University of Science and Technology of China(中国科学技术大学);Institute for Advanced Algorithms Research(高等算法研究院);Northeastern University(东北大学);State Key Laboratory of Media Convergence Production Technology and Systems(媒体融合生产技术与系统国家重点实验室)

Yiquan Wu, Bo Tang, Chenyang Xi, Yu Yu, Pengyu Wang, Yifei Liu, Kun Kuang, Haiying Deng, Zhiyu Li, Feiyu Xiong, Jie Hu, Peng Cheng, Zhonghao Wang, Yi Wang, Yi Luo, Mingchuan Yang

AI总结:

本文提出Xinyu,一个基于LLM的中文评论生成系统,通过将生成过程解构为顺序步骤并结合监督微调满足基本要求,同时利用论点排序模型和检索增强生成(RAG)技术加强证据支撑以满足高级要求,实验证明其有效提升了评论员效率且不牺牲质量。

摘要:

评论通过呈现多样的论点和证据,为读者提供对事件的深刻理解。然而,即使是熟练的评论员,撰写评论也是一项耗时的任务。大型语言模型(LLMs)简化了自然语言生成过程,但由于独特的任务要求,将其直接应用于评论创作仍面临挑战。这些要求可分为两个层面:1)基本要求,包括创建结构良好且逻辑一致的叙述;2)高级要求,涉及生成高质量论点并提供令人信服的证据。在本文中,我们介绍了Xinyu,这是一个高效的基于LLM的系统,旨在辅助评论员生成中文评论。为满足基本要求,我们将生成过程解构为顺序步骤,并为每个步骤提出针对性策略和监督微调(SFT)。为解决高级要求,我们提出了一个用于论点的论点排序模型,并建立了一个包含最新事件和经典书籍的综合证据数据库,从而通过检索增强生成(RAG)技术加强证据的 substantiation。为了更公平地评估生成的评论,对应这两层要求,我们引入了一个综合评估指标,该指标在评论生成中考虑了五个不同的视角。我们的实验证实了所提出系统的有效性。我们还观察到,在真实场景中评论员的效率显著提高,创作评论的平均时间从4小时降至20分钟。重要的是,这种效率的提升并未损害评论的质量。

英文摘要:

Commentary provides readers with a deep understanding of events by presenting diverse arguments and evidence. However, creating commentary is a time-consuming task, even for skilled commentators. Large language models (LLMs) have simplified the process of natural language generation, but their direct application in commentary creation still faces challenges due to unique task requirements. These requirements can be categorized into two levels: 1) fundamental requirements, which include creating well-structured and logically consistent narratives, and 2) advanced requirements, which involve generating quality arguments and providing convincing evidence. In this paper, we introduce Xinyu, an efficient LLM-based system designed to assist commentators in generating Chinese commentaries. To meet the fundamental requirements, we deconstruct the generation process into sequential steps, proposing targeted strategies and supervised fine-tuning (SFT) for each step. To address the advanced requirements, we present an argument ranking model for arguments and establish a comprehensive evidence database that includes up-to-date events and classic books, thereby strengthening the substantiation of the evidence with retrieval augmented generation (RAG) technology. To evaluate the generated commentaries more fairly, corresponding to the two-level requirements, we introduce a comprehensive evaluation metric that considers five distinct perspectives in commentary generation. Our experiments confirm the effectiveness of our proposed system. We also observe a significant increase in the efficiency of commentators in real-world scenarios, with the average time spent on creating a commentary dropping from 4 hours to 20 minutes. Importantly, such an increase in efficiency does not compromise the quality of the commentaries.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑