arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2301.03252cs.CL
收藏

基于主动学习的抽象文本自动摘要

Active Learning for Abstractive Text Summarization

AIRI(AIRI(人工智能研究院));HSE(高等经济学院);SberDevices(SberDevices(俄罗斯联邦储蓄银行设备部));TII(技术创新研究院);Skoltech(斯科尔科沃科技学院);MIPT(莫斯科物理技术学院);MBZUAI(穆罕默德·本·扎耶德人工智能大学);ISP RAS Research Center for Trusted Artificial Intelligence(俄罗斯科学院信息传输问题研究所可信人工智能研究中心)
Akim Tsvigun, Ivan Lysenko, Danila Sedashov, Ivan Lazichny, Eldar Damirov, Vladimir Karlov, Artemy Belousov, Leonid Sanochkin, Maxim Panov, Alexander Panchenko,… 展开作者

Akim Tsvigun, Ivan Lysenko, Danila Sedashov, Ivan Lazichny, Eldar Damirov, Vladimir Karlov, Artemy Belousov, Leonid Sanochkin, Maxim Panov, Alexander Panchenko, Mikhail Burtsev, Artem Shelmanov

摘要:

构建用于抽象文本摘要(ATS)的人工管理的注释数据集非常耗时且昂贵,因为创建每个实例需要人工注释者阅读长文档并撰写较短的摘要,以保留原始文档中的关键信息。主动学习(AL)是一种开发用于减少实现一定水平的机器学习模型性能所需的注释量的技术。在信息抽取和文本分类中,人工智能可以将劳动量减少数倍。尽管它有可能帮助昂贵的注释,据我们所知,没有有效的AL查询策略的ATS。这是因为许多AL策略依赖于不确定性估计,而正如我们在工作中所展示的那样,不确定的实例通常是嘈杂的,与被动注释相比,选择它们会降低模型性能。我们解决这个问题,提出了第一个有效的查询策略AL在ATS的多样性原则的基础上。我们表明,给定一定的注释预算,在AL注释中使用我们的策略有助于提高模型在ROUGE和一致性得分方面的性能。此外,我们分析了自学习的效果,并表明它可以进一步提高模型的性能。

英文摘要:

Construction of human-curated annotated datasets for abstractive text summarization (ATS) is very time-consuming and expensive because creating each instance requires a human annotator to read a long document and compose a shorter summary that would preserve the key information relayed by the original document. Active Learning (AL) is a technique developed to reduce the amount of annotation required to achieve a certain level of machine learning model performance. In information extraction and text classification, AL can reduce the amount of labor up to multiple times. Despite its potential for aiding expensive annotation, as far as we know, there were no effective AL query strategies for ATS. This stems from the fact that many AL strategies rely on uncertainty estimation, while as we show in our work, uncertain instances are usually noisy, and selecting them can degrade the model performance compared to passive annotation. We address this problem by proposing the first effective query strategy for AL in ATS based on diversity principles. We show that given a certain annotation budget, using our strategy in AL annotation helps to improve the model performance in terms of ROUGE and consistency scores. Additionally, we analyze the effect of self-learning and show that it can further increase the performance of the model.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑