分布离散度：一种无条件文本生成的度量
Distributional Discrepancy: A Metric for Unconditional Text Generation
摘要：
无条件文本生成的目的是用真实的句子训练模型，然后生成与训练数据质量相同且多样的新句子。然而，当使用不同的指标来比较无条件文本生成的方法时，得出了相互矛盾的结论。困难在于在评价模型时，样本的多样性和质量应同时考虑。为了解决这个问题，一个新的度量分布差异（DD）的设计，以评估生成器之间的差异生成和真实的训练句子。然而，它不能直接计算DD，因为真实句子的分布不可用。因此，我们提出了一种方法，通过训练基于神经网络的文本分类器来估计DD。为了进行比较，三个现有的指标，双语评估替补（BLEU）与自BLEU，语言模型分数与反向语言模型分数，和Fréchet嵌入距离，以及提出的DD，被用来评估两个流行的生成模型的长短期记忆和生成预训练的Transformer 2的语法和真实数据。实验结果表明，DD是显着优于现有的三个指标，这些生成模型的排名。
英文摘要：
The purpose of unconditional text generation is to train a model with real sentences, then generate novel sentences of the same quality and diversity as the training data. However, when different metrics are used for comparing the methods of unconditional text generation, contradictory conclusions are drawn. The difficulty is that both the diversity and quality of the sample should be considered simultaneously when the models are evaluated. To solve this problem, a novel metric of distributional discrepancy (DD) is designed to evaluate generators based on the discrepancy between the generated and real training sentences. However, it cannot compute the DD directly because the distribution of real sentences is unavailable. Thus, we propose a method for estimating the DD by training a neural-network-based text classifier. For comparison, three existing metrics, bi-lingual evaluation understudy (BLEU) versus self-BLEU, language model score versus reverse language model score, and Fréchet embedding distance, along with the proposed DD, are used to evaluate two popular generative models of long short-term memory and generative pretrained transformer 2 on both syntactic and real data. Experimental results show that DD is significantly better than the three existing metrics for ranking these generative models.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
