arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2104.07219cs.CLcs.AIcs.LG
收藏

多任务处理抑制语义漂移

Multitasking Inhibits Semantic Drift

MIT CSAIL(麻省理工学院计算机科学与人工智能实验室);Facebook AI Research(Facebook人工智能研究院)

Athul Paul Jacob, Mike Lewis, Jacob Andreas

摘要:

当智能体为了实现共同的目标而进行通信时,这些目标如何塑造智能体的语言?我们研究了学习的动态潜在语言政策(LLP),其中教师代理生成自然语言的子目标描述和执行者代理映射这些描述到低级别的行动。LLP可以解决具有挑战性的长期强化学习问题,并为研究面向任务的语言使用提供了丰富的模型。但之前的研究发现,LLP训练容易出现语义漂移(以与原始自然语言含义不一致的方式使用消息)。在这里,我们从理论上和经验上证明了多任务训练是解决这个问题的有效方法:我们证明了多任务训练消除了信号游戏中的语义漂移,并表明在复杂策略游戏中神经LLP的多任务训练减少了漂移,同时提高了样本效率。

英文摘要:

When intelligent agents communicate to accomplish shared goals, how do these goals shape the agents' language? We study the dynamics of learning in latent language policies (LLPs), in which instructor agents generate natural-language subgoal descriptions and executor agents map these descriptions to low-level actions. LLPs can solve challenging long-horizon reinforcement learning problems and provide a rich model for studying task-oriented language use. But previous work has found that LLP training is prone to semantic drift (use of messages in ways inconsistent with their original natural language meanings). Here, we demonstrate theoretically and empirically that multitask training is an effective counter to this problem: we prove that multitask training eliminates semantic drift in a well-studied family of signaling games, and show that multitask training of neural LLPs in a complex strategy game reduces drift and while improving sample efficiency.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑