arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 1909.06618cs.CLcs.LGcs.PF
收藏

数据驱动模型的效率优化:一个文本摘要实例研究

Efficiency Metrics for Data-Driven Models: A Text Summarization Case Study

Institute of Formal and Applied Linguistics, Charles University(查理大学形式与应用语言学研究所)

Erion Çano, Ondřej Bojar

摘要:

使用数据驱动模型来解决文本摘要或类似的任务在过去几年中已经变得非常普遍。然而,大多数研究只报告了基本的准确性分数,并且对所提出的模型在更多数据上训练时的能力一无所知。在本文中,我们定义并提出了三个数据效率指标:数据得分效率,数据时间不足和整体数据效率。我们还提出了一个简单的方案,使用这些指标,并将其应用于文本摘要和标题生成任务的流行方法的更全面的评估。对于后一项任务,我们处理并发布了来自科学文章的3500万个摘要标题对的巨大集合。我们的研究结果表明,在测试模型中,Transformer在这两项任务上都是最有效的。

英文摘要:

Using data-driven models for solving text summarization or similar tasks has become very common in the last years. Yet most of the studies report basic accuracy scores only, and nothing is known about the ability of the proposed models to improve when trained on more data. In this paper, we define and propose three data efficiency metrics: data score efficiency, data time deficiency and overall data efficiency. We also propose a simple scheme that uses those metrics and apply it for a more comprehensive evaluation of popular methods on text summarization and title generation tasks. For the latter task, we process and release a huge collection of 35 million abstract-title pairs from scientific articles. Our results reveal that among the tested models, the Transformer is the most efficient on both tasks.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑