神经文本生成模型的逆向工程实现
Reverse Engineering Configurations of Neural Text Generation Models
摘要：
本文旨在更深入地了解神经文本生成模型的基本特性。对机器生成文本中由于建模选择而出现的伪影的研究是一个新兴的研究领域。以前，这些工件在生成的文本中出现的程度和程度还没有得到很好的研究。本着更好地理解生成文本模型及其工件的精神，我们提出了区分给定模型的几个变体中的哪一个生成了一段文本的新任务，并且我们进行了一系列广泛的诊断测试来观察建模选择（例如，采样方法、前$k$概率、模型架构等）在它们生成的文本中留下可检测的伪像。我们的关键发现，这是由一组严格的实验支持，是这样的工件存在，不同的建模选择可以通过观察生成的文本单独推断。这表明神经文本生成器可能对各种建模选择比以前认为的更敏感。
英文摘要：
This paper seeks to develop a deeper understanding of the fundamental properties of neural text generations models. The study of artifacts that emerge in machine generated text as a result of modeling choices is a nascent research area. Previously, the extent and degree to which these artifacts surface in generated text has not been well studied. In the spirit of better understanding generative text models and their artifacts, we propose the new task of distinguishing which of several variants of a given model generated a piece of text, and we conduct an extensive suite of diagnostic tests to observe whether modeling choices (e.g., sampling methods, top-$k$ probabilities, model architectures, etc.) leave detectable artifacts in the text they generate. Our key finding, which is backed by a rigorous set of experiments, is that such artifacts are present and that different modeling choices can be inferred by observing the generated text alone. This suggests that neural text generators may be more sensitive to various modeling choices than previously thought.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
