arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2210.10329cs.CLcs.AI
收藏

基于属性判别潜在空间的语言去毒化方法

Language Detoxification with Attribute-Discriminative Latent Space

KAIST(韩国科学技术院)

Jin Myung Kwak, Minseon Kim, Sung Ju Hwang

AI总结:

本文提出属性判别语言模型(ADLM),通过将Transformer语言模型的潜在空间投影到属性判别空间,以最小内存和计算开销实现文本去毒化,并在去毒化语言生成和对话生成任务上显著优于基线方法。

摘要:

基于Transformer的语言模型(LMs)在自然语言理解任务上取得了令人瞩目的成果,但它们也可能生成侮辱、威胁和亵渎等有毒文本,从而限制了其在现实世界中的应用。为了解决这一问题,一些文本生成方法尝试利用额外的语言模型或扰动来对有毒文本进行去毒化处理。然而,以往的方法需要过多的内存、计算量和时间,这在其实际应用中构成了严重的瓶颈。为了克服这些局限性,我们提出了一种有效且高效的语言去毒化方法,该方法利用属性判别潜在空间。具体而言,我们通过一个投影模块和一个属性判别器,将原始Transformer语言模型的潜在空间投影到一个判别性潜在空间上,该空间能够根据文本属性将其良好地分离开来。这使得语言模型能够以最小的内存和计算开销来控制文本生成,使其无毒。我们在去毒化语言生成和对话生成任务上验证了我们的模型——属性判别语言模型(ADLM),实验结果表明,我们的方法在性能和效率上都显著优于基线方法。

英文摘要:

Transformer-based Language Models (LMs) have achieved impressive results on natural language understanding tasks, but they can also generate toxic text such as insults, threats, and profanity, limiting their real-world applications. To overcome this issue, a few text generation approaches aim to detoxify toxic texts using additional LMs or perturbations. However, previous methods require excessive memory, computations, and time which are serious bottlenecks in their real-world application. To address such limitations, we propose an effective yet efficient method for language detoxification using an attribute-discriminative latent space. Specifically, we project the latent space of an original Transformer LM onto a discriminative latent space that well-separates texts by their attributes using a projection block and an attribute discriminator. This allows the LM to control the text generation to be non-toxic with minimal memory and computation overhead. We validate our model, Attribute-Discriminative Language Model (ADLM) on detoxified language and dialogue generation tasks, on which our method significantly outperforms baselines both in performance and efficiency.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑