arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2302.11875cs.CL
收藏

改进的混合专家模型语言GANs训练

Improved Training of Mixture-of-Experts Language GANs

Baidu(百度);Institute of Automation, CAS(中国科学院自动化研究所)

Yekun Chai, Qiyue Yin, Junge Zhang

摘要:

尽管在图像生成方面取得了巨大的成功,但生成对抗网络(GANs)在合成离散元素序列(特别是人类语言)方面仍然面临巨大挑战。生成器训练的困难来自于有限的表示能力和从训练器获得的无信息学习信号。在这项工作中,我们(1)首先经验性地表明,混合专家模型方法能够增强语言GANs生成器的表示能力,(2)利用特征统计对齐(FSA)范式来呈现细粒度的学习信号以推进生成器训练。具体地说,FSA迫使假数据的分布的平均统计量在有限维特征空间中尽可能接近真实样本的分布。在合成和真实基准测试上的实验研究表明了该方法在定量评估方面的优越性能,并证明了该方法在对抗性文本生成中的有效性。

英文摘要:

Despite the dramatic success in image generation, Generative Adversarial Networks (GANs) still face great challenges in synthesizing sequences of discrete elements, in particular human language. The difficulty in generator training arises from the limited representation capacity and uninformative learning signals obtained from the discriminator. In this work, we (1) first empirically show that the mixture-of-experts approach is able to enhance the representation capacity of the generator for language GANs and (2) harness the Feature Statistics Alignment (FSA) paradigm to render fine-grained learning signals to advance the generator training. Specifically, FSA forces the mean statistics of the distribution of fake data to approach that of real samples as close as possible in the finite-dimensional feature space. Empirical study on synthetic and real benchmarks shows the superior performance in quantitative evaluation and demonstrates the effectiveness of our approach to adversarial text generation.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑