结构预测的Softmax Q分布估计:对RAML的理论解释
Softmax Q-Distribution Estimation for Structured Prediction: A Theoretical Interpretation for RAML
摘要:
奖励增强最大似然(RAML),一个简单而有效的学习框架,直接优化结构化预测任务中的奖励函数,已经取得了许多令人印象深刻的经验成功。RAML通过对根据指数回报分布采样的候选输出执行最大似然更新来合并特定于任务的奖励,这为接近参考输出的候选者提供了更高的概率。虽然RAML以其简单、高效和令人印象深刻的经验成功而闻名,但RAML的理论性质,特别是指数化收益分布的行为,还没有得到彻底的研究。在这项工作中,我们介绍了软最大Q分布估计,一种新的理论解释RAML,揭示了RAML和贝叶斯决策理论之间的关系。softmax Q分布可以看作是贝叶斯决策边界的一个光滑近似,贝叶斯决策规则可以通过这个Q分布进行译码得到。我们进一步表明,RAML是等效的近似估计softmax Q分布,与温度$τ$控制近似误差。我们进行了两个实验,一个是多类分类的合成数据和一个真实的图像描述的数据,以证明RAML和所提出的softmax Q分布估计方法之间的关系,验证我们的理论分析。在三个结构化预测任务上进行的额外实验表明,与最大似然基线相比,在顺序(命名实体识别)、基于树的(依存句法分析)和不规则(机器翻译)结构上定义了奖励。
英文摘要:
Reward augmented maximum likelihood (RAML), a simple and effective learning framework to directly optimize towards the reward function in structured prediction tasks, has led to a number of impressive empirical successes. RAML incorporates task-specific reward by performing maximum-likelihood updates on candidate outputs sampled according to an exponentiated payoff distribution, which gives higher probabilities to candidates that are close to the reference output. While RAML is notable for its simplicity, efficiency, and its impressive empirical successes, the theoretical properties of RAML, especially the behavior of the exponentiated payoff distribution, has not been examined thoroughly. In this work, we introduce softmax Q-distribution estimation, a novel theoretical interpretation of RAML, which reveals the relation between RAML and Bayesian decision theory. The softmax Q-distribution can be regarded as a smooth approximation of the Bayes decision boundary, and the Bayes decision rule is achieved by decoding with this Q-distribution. We further show that RAML is equivalent to approximately estimating the softmax Q-distribution, with the temperature $τ$ controlling approximation error. We perform two experiments, one on synthetic data of multi-class classification and one on real data of image captioning, to demonstrate the relationship between RAML and the proposed softmax Q-distribution estimation method, verifying our theoretical analysis. Additional experiments on three structured prediction tasks with rewards defined on sequential (named entity recognition), tree-based (dependency parsing) and irregular (machine translation) structures show notable improvements over maximum likelihood baselines.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。