IFDID：基于多样性增强解码的信息过滤器，用于自然语言生成中的多样性-忠实度权衡
IFDID: Information Filter upon Diversity-Improved Decoding for Diversity-Faithfulness Tradeoff in NLG
AI总结：
本文提出两阶段IFDID解码策略，通过Enhance-Filter框架提升典型token概率并按信息量过滤，在中英文基准上同时改善忠实性与多样性，达到SOTA权衡效果。
摘要：
一些自然语言生成（NLG）任务同时要求忠实性和多样性。解码策略与生成文本的质量密切相关。beam search、greedy search等策略表现出低多样性和高重复性。另一方面，作为多样性解决方案的引导式解码可能会生成不忠实的表达。为此，本文提出基于多样性增强解码的信息过滤器（Information Filter upon Diversity-Improved Decoding，IFDID），以获得多样性与忠实性之间的权衡。IFDID是一种两阶段解码策略，利用所提出的Enhance-Filter框架，通过提高某些典型token被选中的概率，随后根据其信息量对其进行过滤，从而实现这种权衡。为验证有效性，我们在相关的CommonGEN、RocStories和AdGen基准上将本文方法与其他基线进行比较，这些基准涵盖中文和英文数据集。我们的数值实验结果和人工评估结果验证了所提方法的有效性：本方法在描述忠实性的ROUGE分数上高出1.24，并且在以Dist-2表示的多样性上比传统方法高出62.5%，表明IFDID是一种用于多样性与忠实性权衡的新型SOTA解码策略。
英文摘要：
Some Natural Language Generation (NLG) tasks require both faithfulness and diversity. The decoding strategy is intensively related to the quality of the generated text. Strategies such as beam search, greedy search, etc., perform with low diversity and high repetition. On the other hand, guided decoding, the solution towards diversity, may generate unfaithful expressions. To this end, this paper presents Information Filter upon Diversity-Improved Decoding (IFDID) to obtain the tradeoff between diversity and faithfulness. IFDID is a two-stage decoding strategy leveraging the proposed Enhance-Filter framework, which achieves the tradeoff by increasing the probabilities of some typical tokens being selected and subsequently filtering them by their information amount. To verify the effectiveness, we compare our method with other baselines on related CommonGEN, RocStories and AdGen benchmarks, which cover Chinese and English datasets. Our numerical experimental results and human evaluation outcomes verify the effectiveness of the proposed approach, as our approach achieves a 1.24 higher ROUGE score describing faithfulness as well as higher diversity represented by 62.5% higher upon Dist-2 than traditional approaches, demonstrating that IFDID is a novel SOTA decoding strategy for the tradeoff between diversity and faithfulness.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
