arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2302.08577cs.CL
收藏

对于生成文本,NLI中性文本是最佳文本吗?

For Generated Text, Is NLI-Neutral Text the Best Text?

The University of Texas at Austin(德克萨斯大学奥斯汀分校)

Michail Mersinias, Kyle Mahowald

AI总结:

本文利用预训练NLI模型检测生成句与提示及前文的推理关系,并为GPT-J设计NLI引导生成策略,实验证明最大化中性类别可稳定获得最高文本质量。

摘要:

我们探索将自然语言推理(NLI)融入文本生成流程,方法是使用预训练NLI模型评估生成句子相对于提示及前文是蕴含、矛盾还是中立。首先,我们表明NLI任务能够预测GPT-3所犯的生成错误。我们利用这些结果为GPT-J开发了一种结合NLI信息的生成流程。随后,我们通过获取关于错误类型和整体质量的人工标注来评估这些生成结果。我们发现,当nucleus sampling随机性参数值较高时,最大化蕴含的NLI策略能够改善文本生成;而当该参数值较低时,最大化矛盾的策略实际上反而有效。不过总体而言,我们证明,无论参数值如何,最大化中性类别的NLI策略都能产生最高质量的生成文本(显著优于普通生成结果)。

英文摘要:

We explore incorporating natural language inference (NLI) into the text generative pipeline by using a pre-trained NLI model to assess whether a generated sentence entails, contradicts, or is neutral to the prompt and preceding text. First, we show that the NLI task is predictive of generation errors made by GPT-3. We use these results to develop an NLI-informed generation procedure for GPT-J. Then, we evaluate these generations by obtaining human annotations on error types and overall quality. We find that an NLI strategy of maximizing entailment improves text generation when the nucleus sampling randomness parameter value is high, while one which maximizes contradiction is in fact productive when the parameter value is low. Overall, though, we demonstrate that an NLI strategy of maximizing the neutral class provides the highest quality of generated text (significantly better than the vanilla generations), regardless of parameter value.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑