arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2204.09716cs.CLcs.AI
收藏

用于自然语言摘要的去噪序列到序列模型的领域特定微调

Domain Specific Fine-tuning of Denoising Sequence-to-Sequence Models for Natural Language Summarization

Deloitte(德勤);University of Toronto(多伦多大学);Western University(韦仕敦大学);PwC Canada(普华永道加拿大);Vector Institute(向量研究所)

Brydon Parker, Alik Sokolov, Mahtab Ahmed, Matt Kalebic, Sedef Akinli Kocak, Ofer Shai

摘要:

摘要的长形式的文本数据是一个问题,特别是在知识经济的工作,如医药和金融,需要不断保持了解一个复杂的和不断发展的知识体系。因此,使用自然语言处理(NLP)技术自动隔离和汇总关键内容,在这些行业中具有节省大量时间的潜力。我们探索了最先进的NLP模型(BART)的应用,并探索了使用数据增强和各种微调策略将其调整到最佳性能的策略。我们表明,当对特定领域的数据进行测试时,我们的端到端微调方法可以使ROUGE-1比开箱即用的预训练BART摘要器提高5-6\%,并使我们的端到端管道可以在金融,医疗或其他用户指定的领域实现这些结果。

英文摘要:

Summarization of long-form text data is a problem especially pertinent in knowledge economy jobs such as medicine and finance, that require continuously remaining informed on a sophisticated and evolving body of knowledge. As such, isolating and summarizing key content automatically using Natural Language Processing (NLP) techniques holds the potential for extensive time savings in these industries. We explore applications of a state-of-the-art NLP model (BART), and explore strategies for tuning it to optimal performance using data augmentation and various fine-tuning strategies. We show that our end-to-end fine-tuning approach can result in a 5-6\% absolute ROUGE-1 improvement over an out-of-the-box pre-trained BART summarizer when tested on domain specific data, and make available our end-to-end pipeline to achieve these results on finance, medical, or other user-specified domains.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑