arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2109.02297cs.CLcs.AI
收藏

基于语义的策略学习和增强猜测增强视觉对话提问者

Enhancing Visual Dialog Questioner with Entity-based Strategy Learning and Augmented Guesser

Beijing University of Posts and Telecommunications(北京邮电大学);Pattern Recognition Center, WeChat AI, Tencent, China(腾讯微信AI模式识别中心);Soochow University(苏州大学)

Duo Zheng, Zipeng Xu, Fandong Meng, Xiaojie Wang, Jiaan Wang, Jie Zhou

摘要:

考虑到建立一个良好的视觉对话(VD)提问者的重要性,许多研究者在Q-Bot-A-Bot图像猜测游戏的设置下研究该主题,其中提问者需要提出一系列问题来收集未公开图像的信息。尽管监督学习(SL)和强化学习(RL)已经取得了进展,但仍然存在问题。首先,以前的方法没有为提问者提供明确有效的指导,以产生视觉相关和信息丰富的问题。其次,RL的效果受到不合格成分的阻碍,即,猜测者,他根据生成的对话做出图像预测,并相应地分配奖励。要增强VD怀疑者:1)我们提出了一个相关实体增强的提问者(ReeQ),它在相关实体的指导下生成问题,并从人类对话中学习基于实体的提问策略; 2)我们提出了一个增强的猜测者(AugG),它是强大的,特别是针对VD设置进行了优化。在VisDial v1.0数据集上的实验结果表明,该方法在图像猜测任务和问题多样性上都达到了最佳性能。人类研究进一步证明,我们的模型产生更多的视觉相关,信息和连贯的问题。

英文摘要:

Considering the importance of building a good Visual Dialog (VD) Questioner, many researchers study the topic under a Q-Bot-A-Bot image-guessing game setting, where the Questioner needs to raise a series of questions to collect information of an undisclosed image. Despite progress has been made in Supervised Learning (SL) and Reinforcement Learning (RL), issues still exist. Firstly, previous methods do not provide explicit and effective guidance for Questioner to generate visually related and informative questions. Secondly, the effect of RL is hampered by an incompetent component, i.e., the Guesser, who makes image predictions based on the generated dialogs and assigns rewards accordingly. To enhance VD Questioner: 1) we propose a Related entity enhanced Questioner (ReeQ) that generates questions under the guidance of related entities and learns entity-based questioning strategy from human dialogs; 2) we propose an Augmented Guesser (AugG) that is strong and is optimized for the VD setting especially. Experimental results on the VisDial v1.0 dataset show that our approach achieves state-of-theart performance on both image-guessing task and question diversity. Human study further proves that our model generates more visually related, informative and coherent questions.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑