arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2110.04400cs.CL
收藏

HydraSum:使用多解码器模型在文本摘要中分解文体特征

HydraSum: Disentangling Stylistic Features in Text Summarization using Multi-Decoder Models

The University of Texas at Austin(德克萨斯大学奥斯汀分校);Hugging Face(Hugging Face(抱抱脸));Faire;Salesforce Research(赛富时研究院)

Tanya Goyal, Nazneen Fatema Rajani, Wenhao Liu, Wojciech Kryściński

摘要:

摘要系统在推理过程中对摘要属性做出了许多“决定”,例如复制程度、输出的特异性和长度等。然而,这些都被隐式地编码在模型参数中,无法强制执行特定的样式。为了解决这个问题,我们引入了HydraSum,这是一种新的摘要架构,它将当前模型的单个解码器框架扩展到具有多个解码器的专家混合版本。我们表明,HydraSum的多个解码器自动学习对比摘要风格时,在标准的训练目标,没有任何额外的监督下训练。通过对三个摘要数据集(CNN,Newsroom和XSum)的实验,我们表明HydraSum提供了一种简单的机制,通过从单个解码器或其混合物中采样来获得风格多样的摘要,优于基线模型。最后,我们证明了在训练过程中对门控策略的一个小的修改可以强制执行更严格的风格划分,例如高与低抽象性或高与低特异性,允许用户从生成空间中的更大区域进行采样,并沿多个维度改变摘要风格。

英文摘要:

Summarization systems make numerous "decisions" about summary properties during inference, e.g. degree of copying, specificity and length of outputs, etc. However, these are implicitly encoded within model parameters and specific styles cannot be enforced. To address this, we introduce HydraSum, a new summarization architecture that extends the single decoder framework of current models to a mixture-of-experts version with multiple decoders. We show that HydraSum's multiple decoders automatically learn contrasting summary styles when trained under the standard training objective without any extra supervision. Through experiments on three summarization datasets (CNN, Newsroom and XSum), we show that HydraSum provides a simple mechanism to obtain stylistically-diverse summaries by sampling from either individual decoders or their mixtures, outperforming baseline models. Finally, we demonstrate that a small modification to the gating strategy during training can enforce an even stricter style partitioning, e.g. high- vs low-abstractiveness or high- vs low-specificity, allowing users to sample from a larger area in the generation space and vary summary styles along multiple dimensions.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑