arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2306.09607cs.CL
收藏

面向PhotoBook指称游戏的、以CLIPScores作为隐式指称链的听者模型

Listener Model for the PhotoBook Referential Game with CLIPScores as Implicit Reference Chain

Carnegie Mellon University(卡内基梅隆大学)

Shih-Lun Wu, Yi-Hui Chou, Liangze Li

AI总结:

针对PhotoBook协作对话游戏现有方法依赖不完善的指称链、无法落地实际游戏的问题,提出基于DeBERTa、利用CLIPScore评估图文相关性的无指称链听者模型,在未见过的主题上准确率超77%,显著优于基线。

摘要:

PhotoBook是一款协作对话游戏,两名玩家会收到各自私有的、部分重叠的图像集,并需要找出双方共有的图像。它给机器带来了一项重大挑战:学习人类如何围绕多模态语境建立共同认知以实现有效沟通。然而,现有文献中提出的方法无法应用于实际游戏流程,因为它们仅能处理该游戏的部分子任务,且需要额外的指称链输入,而指称链的提取过程并不完善。因此,我们提出了一种无需指称链的听者模型,直接解决该游戏的预测任务,即判断某张图像是否为与搭档共有的图像。我们的基于DeBERTa的听者模型会读取完整对话,并利用CLIPScore特征评估话语与图像的相关性。我们在未见过的图像集/游戏主题上取得了超过77%的准确率,比基线模型高出17个百分点以上。

英文摘要:

PhotoBook is a collaborative dialogue game where two players receive private, partially-overlapping sets of images and resolve which images they have in common. It presents machines with a great challenge to learn how people build common ground around multimodal context to communicate effectively. Methods developed in the literature, however, cannot be deployed to real gameplay since they only tackle some subtasks of the game, and they require additional reference chains inputs, whose extraction process is imperfect. Therefore, we propose a reference chain-free listener model that directly addresses the game's predictive task, i.e., deciding whether an image is shared with partner. Our DeBERTa-based listener model reads the full dialogue, and utilizes CLIPScore features to assess utterance-image relevance. We achieve >77% accuracy on unseen sets of images/game themes, outperforming baseline by >17 points.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑