发表机构
Lancaster University（兰卡斯特大学）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
本研究针对伦敦证券交易所上市公司年度报告的长文档摘要任务，比较了多种预训练Transformer模型和提取技术，提出BRUGEscore指标（ROUGE-2与BERTscore的调和平均）以改进评估，并通过统计检验和对抗性分析验证了其稳健性。
AI 中文摘要
英国伦敦证券交易所拥有超过2000家上市公司，分为11个行业，这些公司被要求在每个财政年度内至少两次传达其财务业绩。英国年度报告是非常冗长的文件，平均约80页。在本研究中，我们旨在针对不同的预训练Transformer模型，结合不同的提取技术，对各种摘要方法进行基准测试。此外，我们考虑了多种评估指标，以研究它们在来自金融叙事摘要（FNS 2020）共享任务的数据集上的不同行为和适用性，该数据集由伦敦证券交易所上市公司发布的年度报告及其对应摘要组成。我们假设一些评估指标不能反映真正的摘要能力，并提出了一种新颖的BRUGEscore指标，作为ROUGE-2和BERTscore的调和平均值。最后，我们对结果进行了统计显著性检验，以验证其统计稳健性，并进行了包含三种不同损坏方法的对抗性分析任务。
英文摘要
There are more than 2,000 listed companies on the UK's London Stock Exchange, divided into 11 sectors who are required to communicate their financial results at least twice in a single financial year. UK annual reports are very lengthy documents with around 80 pages on average. In this study, we aim to benchmark a variety of summarisation methods on a set of different pre-trained transformers with different extraction techniques. In addition, we considered multiple evaluation metrics in order to investigate their differing behaviour and applicability on a dataset from the Financial Narrative Summarisation (FNS 2020) shared task, which is composed of annual reports published by firms listed on the London Stock Exchange and their corresponding summaries. We hypothesise that some evaluation metrics do not reflect true summarisation ability and propose a novel BRUGEscore metric, as the harmonic mean of ROUGE-2 and BERTscore. Finally, we perform a statistical significance test on our results to verify whether they are statistically robust, alongside an adversarial analysis task with three different corruption methods.
Comments12 pages
Journal refNatural Language Processing and Information Systems. NLDB 2023
DOI：10.1007/978-3-031-35320-8_28
