改进IRGAN收敛性的近端策略优化
Proximal Policy Optimization for Improved Convergence in IRGAN
摘要:
IRGAN是一种信息检索(IR)建模方法,它利用生成模型和判别模型之间的理论极小极大博弈来迭代优化生成模型和判别模型,从而统一了生成模型和判别模型。尽管在几个信息检索任务中有显着的性能改进,但IRGAN训练是一个不稳定的过程,并且解决方案随着随机参数初始化而变化很大。在这项工作中,我们提出了一个改进的训练目标的基础上,近端策略优化目标和Gumbel-Softmax为基础的采样生成器。我们还提出了一个修改后的训练算法,它需要一个单一的梯度更新的生成器以及每个迭代步骤的迭代。我们提出了改进的收敛性的实证证据,该模型在原来的IRGAN和三个不同的IR任务的基准数据集上的比较也进行了讨论,强调该模型的优越性能。
英文摘要:
IRGAN is an information retrieval (IR) modeling approach that uses a theoretical minimax game between a generative and a discriminative model to iteratively optimize both of them, hence unifying the generative and discriminative approaches. Despite significant performance improvements in several information retrieval tasks, IRGAN training is an unstable process, and the solution varies largely with the random parameter initialization. In this work, we present an improved training objective based on proximal policy optimization objective and Gumbel-Softmax based sampling for the generator. We also propose a modified training algorithm which takes a single gradient update on both the generator as well as discriminator for each iteration step. We present empirical evidence of the improved convergence of the proposed model over the original IRGAN and a comparison on three different IR tasks on benchmark datasets is also discussed, emphasizing the proposed model's superior performance.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。