To engage in human-like dialogue, robots require the ability to describe the objects, locations, and people in their environment, a capability known as "Referring Expression Generation." As speakers repeatedly refer to similar objects, they tend to re-use properties from previous descriptions, in part to help the listener, and in part due to cognitive availability of those properties in working memory (WM). Because different theories of working memory "forgetting" necessarily lead to differences in cognitive availability, we hypothesize that they will similarly result in generation of different referring expressions. To design effective intelligent agents, it is thus necessary to determine how different models of forgetting may be differentially effective at producing natural human-like referring expressions. In this work, we computationalize two candidate models of working memory forgetting within a robot cognitive architecture, and demonstrate how they lead to cognitive availability-based differences in generated referring expressions.
目前的指称理解算法能够有效地检测或分割名词所指的对象，但如何理解动词指称仍然是一个有待研究的问题。因此，我们研究了具有挑战性的问题，面向任务的检测，其目的是找到对象，最能承受的动词，如sit comfortable on. Towards一个更好的本地化，更好地服务于下游的应用程序，如机器人交互，我们扩展到面向任务的实例分割的问题。这项任务的一个独特要求是在可能的备选方案中选择首选的候选方案。因此，我们求助于Transformer架构，它自然地对关注的查询关系进行建模，从而产生TOIST方法。为了利用预先训练的名词指称表达理解模型以及我们可以在训练期间访问特权名词基础事实的事实，提出了一种新的名词-代词蒸馏框架。名词原型以无监督的方式生成，并训练上下文代词特征来选择原型。因此，网络在推理过程中保持名词不可知。我们在大规模面向任务的数据集COCO-Tasks上评估TOIST，并实现了比最佳报告结果高出$\rm{mAP^{box}}$ 10.9%的结果。所提出的名词-代词蒸馏可以将$\rm{mAP^{box}}$和$\rm{mAP^{mask}}$提高+2.8%和+3.8%。代码和模型可在https://github.com/AIR-DISCOVER/TOIST上公开获取。
