arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2109.09390cs.AI
收藏

社会监督下的表示学习:主体性在学习有效表示中的作用

Socially Supervised Representation Learning: the Role of Subjectivity in Learning Efficient Representations

Inria(法国国家信息与自动化研究所);Université de Bordeaux(波尔多大学);ENSTA ParisTech(巴黎高等先进技术学院)

Julius Taylor, Eleni Nisioti, Clément Moulin-Frier

摘要:

尽管自监督学习已经成为解决当今机器学习模型数据效率低下的一个突出解决方案,但它还没有从纯粹的多智能体角度进行研究。在这项工作中,我们提出,对齐内部主观表示,这自然会出现在多代理设置,代理接收相同的底层环境状态的部分观察,可以导致更有效的数据表示。我们建议,多代理环境,代理人没有访问他人的意见,但可以在有限的范围内进行通信,保证一个共同的背景下,可以利用在个人表示学习。原因在于,主观观察必然涉及潜在环境状态的相同子集,并且关于这些状态的通信可以自由地提供监督信号。为了强调沟通的重要性,我们将我们的设置称为\textit{socially supervised representation learning}。我们提出了一个最小的架构组成的人口的自动编码器,在那里我们定义损失函数,捕捉不同方面的有效沟通,并检查其效果的学习表示。我们表明,我们提出的架构允许对齐表示的出现。通过呈现具有环境状态的不同视角的代理所引入的主观性有助于学习抽象表示,这些抽象表示优于由单个自动编码器和一群自动编码器所学习到的抽象表示,这些自动编码器以相同的环境状态视角呈现。总之,我们的研究结果表明,从主观角度的通信可以导致收购更抽象的表示在多智能体系统,开放的前景为未来的研究在交叉点的表示学习和紧急通信。

英文摘要:

Despite its rise as a prominent solution to the data inefficiency of today's machine learning models, self-supervised learning has yet to be studied from a purely multi-agent perspective. In this work, we propose that aligning internal subjective representations, which naturally arise in a multi-agent setup where agents receive partial observations of the same underlying environmental state, can lead to more data-efficient representations. We propose that multi-agent environments, where agents do not have access to the observations of others but can communicate within a limited range, guarantees a common context that can be leveraged in individual representation learning. The reason is that subjective observations necessarily refer to the same subset of the underlying environmental states and that communication about these states can freely offer a supervised signal. To highlight the importance of communication, we refer to our setting as \textit{socially supervised representation learning}. We present a minimal architecture comprised of a population of autoencoders, where we define loss functions, capturing different aspects of effective communication, and examine their effect on the learned representations. We show that our proposed architecture allows the emergence of aligned representations. The subjectivity introduced by presenting agents with distinct perspectives of the environment state contributes to learning abstract representations that outperform those learned by a single autoencoder and a population of autoencoders, presented with identical perspectives of the environment state. Altogether, our results demonstrate how communication from subjective perspectives can lead to the acquisition of more abstract representations in multi-agent systems, opening promising perspectives for future research at the intersection of representation learning and emergent communication.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑