arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2108.03502cs.CL
收藏

俄语文本摘要的GPT-3微调

Fine-tuning GPT-3 for Russian Text Summarization

Federal State Budget Educational Institution of Higher Education «MIREA - Russian Technological University»(联邦国家预算高等教育机构「MIREA——俄罗斯技术大学」);National Research University Higher School of Economics(国立研究型大学高等经济学院)

Alexandr Nikolich, Arina Puchkova

摘要:

自动摘要技术旨在缩短和概括文本中给出的信息,同时保留其核心信息和最相关的想法。这个任务可以用各种方法来处理和处理,然而,尽管现有的最先进的模型的本地化,但并没有进行很多尝试来产生专门针对俄语的解决方案。在本文中,我们的目标是展示ruGPT3的能力,总结文本,微调它对语料库的俄罗斯新闻与相应的人工生成的摘要。此外,我们采用超参数调整,使模型的输出变得不那么随机,更紧密地联系到原始文本。我们评估所产生的文本与一组指标,表明我们的解决方案可以超越国家的最先进的模型的性能,而无需额外的架构或损失函数的变化。尽管能够生成合理的摘要,但我们的模型仍然存在一些缺陷,即它容易改变原始文本中存在的命名实体(如姓氏,地点,日期),偏离给定文档中陈述的事实,并重复摘要中的信息。

英文摘要:

Automatic summarization techniques aim to shorten and generalize information given in the text while preserving its core message and the most relevant ideas. This task can be approached and treated with a variety of methods, however, not many attempts have been made to produce solutions specifically for the Russian language despite existing localizations of the state-of-the-art models. In this paper, we aim to showcase ruGPT3 ability to summarize texts, fine-tuning it on the corpora of Russian news with their corresponding human-generated summaries. Additionally, we employ hyperparameter tuning so that the model's output becomes less random and more tied to the original text. We evaluate the resulting texts with a set of metrics, showing that our solution can surpass the state-of-the-art model's performance without additional changes in architecture or loss function. Despite being able to produce sensible summaries, our model still suffers from a number of flaws, namely, it is prone to altering Named Entities present in the original text (such as surnames, places, dates), deviating from facts stated in the given document, and repeating the information in the summary.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑