arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2502.17136cs.AIcs.IR
收藏

评估大型语言模型在自动化新闻文章摘要中的有效性

Evaluating the Effectiveness of Large Language Models in Automated News Article Summarization

Institute of Computer Science University of Applied Sciences Ruhr West Mülheim an der Ruhr, Germany

Lionel Richy Panlap Houamegni, Fatih Gedikli

AI总结:

本研究利用大型语言模型构建自动化新闻摘要系统,用于供应链风险分析,验证了LLMs(尤其是Few-Shot GPT-4o mini)在摘要质量和风险识别上的显著有效性。

摘要:

新闻分析与摘要的自动化为处理和分析当今信息社会中普遍存在的大量信息这一挑战提供了一种有前景的解决方案。大型语言模型(LLMs)已展现出将大量文本数据转化为简洁且易于理解的摘要的能力,为信息过载问题提供了有效的解决方案,并为用户提供了相关信息的快速概览。该技术的一个特别重要的应用在于供应链风险分析。公司必须监控其供应商的新闻并对事件做出响应,原因包括遵守法律法规、风险管理以及保持供应链韧性等几个关键方面。本文开发了一个使用LLMs进行供应链风险分析的自动化新闻摘要系统。所提出的解决方案聚合来自各种来源的新闻,使用LLMs对其进行摘要,并以清晰简洁的格式向用户呈现浓缩信息。这种方法使公司能够优化其信息处理并做出明智的决策。我们的研究解决了两个主要研究问题:(1)LLMs在自动化新闻摘要方面是否有效,特别是在供应链风险分析的背景下?(2)各种LLMs在摘要质量方面的可读性、重复检测和风险识别方面效果如何?在本文中,我们使用当时一系列公开可用的LLMs进行了一项离线研究,并辅以一项针对离线实验中表现最佳系统的用户研究,以进一步评估其有效性。我们的结果表明,LLMs,特别是Few-Shot GPT-4o mini,在摘要质量和风险识别方面提供了显著的改进。

英文摘要:

The automation of news analysis and summarization presents a promising solution to the challenge of processing and analyzing vast amounts of information prevalent in today's information society. Large Language Models (LLMs) have demonstrated the capability to transform vast amounts of textual data into concise and easily comprehensible summaries, offering an effective solution to the problem of information overload and providing users with a quick overview of relevant information. A particularly significant application of this technology lies in supply chain risk analysis. Companies must monitor the news about their suppliers and respond to incidents for several critical reasons, including compliance with laws and regulations, risk management, and maintaining supply chain resilience. This paper develops an automated news summarization system for supply chain risk analysis using LLMs. The proposed solution aggregates news from various sources, summarizes them using LLMs, and presents the condensed information to users in a clear and concise format. This approach enables companies to optimize their information processing and make informed decisions. Our study addresses two main research questions: (1) Are LLMs effective in automating news summarization, particularly in the context of supply chain risk analysis? (2) How effective are various LLMs in terms of readability, duplicate detection, and risk identification in their summarization quality? In this paper, we conducted an offline study using a range of publicly available LLMs at the time and complemented it with a user study focused on the top performing systems of the offline experiments to evaluate their effectiveness further. Our results demonstrate that LLMs, particularly Few-Shot GPT-4o mini, offer significant improvements in summary quality and risk identification.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑