arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2402.04824cs.CL
收藏

学习协作参考游戏中针对不同跟随者行为的通信策略

Learning Communication Policies for Different Follower Behaviors in a Collaborative Reference Game

University of Potsdam(波茨坦大学);German Research Center for Artificial Intelligence (DFKI)(德国人工智能研究中心 (DFKI))

Philipp Sadler, Sherzod Hakimov, David Schlangen

AI总结:

本研究在协作参考游戏中,通过强化学习训练指导者智能体,并引入沟通努力信号,使其策略适应不同置信度和自主性的跟随者行为,从而产生更简洁的沟通策略。

摘要:

Albrecht和Stone(2018)指出,对其他智能体可能采取的行为缺乏本质约束,使得对变化行为的建模仍然是一个开放问题。在本工作中,我们评估了神经智能体在协作参考游戏中对假定伙伴行为的适应性。在该游戏中,当知识渊博的指导者能够通过口头引导跟随者从多个干扰项中选择特定拼图块时,即视为成功。我们将这一语言基础与协调任务构建为强化学习问题,并衡量一种常见的强化训练算法(PPO)在多大程度上能够产生与各种沿置信度和自主性维度变化的启发式跟随者行为表现良好的神经智能体(指导者)。我们实验了一种学习信号,该信号除目标条件外还考虑了假定的沟通努力。我们的结果表明,这一新颖成分导致了更简洁的沟通策略(在某些步骤中保持沉默),并且在这方面,指导者的策略确实适应了伙伴的置信度和自主性水平。

英文摘要:

Albrecht and Stone (2018) state that modeling of changing behaviors remains an open problem "due to the essentially unconstrained nature of what other agents may do". In this work we evaluate the adaptability of neural artificial agents towards assumed partner behaviors in a collaborative reference game. In this game success is achieved when a knowledgeable Guide can verbally lead a Follower to the selection of a specific puzzle piece among several distractors. We frame this language grounding and coordination task as a reinforcement learning problem and measure to which extent a common reinforcement training algorithm (PPO) is able to produce neural agents (the Guides) that perform well with various heuristic Follower behaviors that vary along the dimensions of confidence and autonomy. We experiment with a learning signal that in addition to the goal condition also respects an assumed communicative effort. Our results indicate that this novel ingredient leads to communicative strategies that are less verbose (staying silent in some of the steps) and that with respect to that the Guide's strategies indeed adapt to the partner's level of confidence and autonomy.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑