ETC-NLG:端到端主题条件自然语言生成
ETC-NLG: End-to-end Topic-Conditioned Natural Language Generation
AI总结:
提出ETC-NLG方法,利用主题建模标注在无标签文档集合上实现完全无监督的端到端主题条件自然语言生成,并在低资源场景下验证其有效性。
摘要:
即插即用语言模型(PPLMs)通过将大型预训练生成器与用于将预测词元分布引导至所选主题的属性模型配对,实现了主题条件自然语言生成。尽管具有计算效率,PPLMs仍需大量标注文本以有效平衡生成流畅性与适当条件化,使其不适用于低资源场景。我们提出ETC-NLG,该方法利用主题建模标注,在无标签文档集合中的涌现主题上实现完全无监督的端到端主题条件自然语言生成。我们首先在意大利语低资源场景中测试该方法的有效性,评估主题模型与黄金标注的条件化效果。随后使用平行语料库对意大利语和英语的ETC-NLG进行对比评估。最后,我们提出一种自动方法来估计对生成话语条件化的有效性。
英文摘要:
Plug-and-play language models (PPLMs) enable topic-conditioned natural language generation by pairing large pre-trained generators with attribute models used to steer the predicted token distribution towards the selected topic. Despite their computational efficiency, PPLMs require large amounts of labeled texts to effectively balance generation fluency and proper conditioning, making them unsuitable for low-resource settings. We present ETC-NLG, an approach leveraging topic modeling annotations to enable fully-unsupervised End-to-end Topic-Conditioned Natural Language Generation over emergent topics in unlabeled document collections. We first test the effectiveness of our approach in a low-resource setting for Italian, evaluating the conditioning for both topic models and gold annotations. We then perform a comparative evaluation of ETC-NLG for Italian and English using a parallel corpus. Finally, we propose an automatic approach to estimate the effectiveness of conditioning on the generated utterances.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。