通过率失真理论视角探索以行动为中心的表征
Exploring Action-Centric Representations Through the Lens of Rate-Distortion Theory
AI总结:
该研究基于率失真理论探索以行动为中心的表征,给出抽象的数学定义,用VAE建模发现这类表征是高效有损压缩、具备任务依赖不变性且不服务于数据重构,支持感知的目的论观点。
摘要:
生物体必须持续追踪环境中与适应性行为相关的信息。对于生存于高维环境、资源有限的智能体而言,以经济高效的方式传输信息变得至关重要。高效编码假说主张,生物体会寻求以高效的方式最大化关于感官输入的信息。在贝叶斯推理框架下,这意味着大脑的作用是高效分配资源,以对引发感官数据的隐藏状态做出预测。然而,这两种框架均未说明这些信息如何被下游系统利用,忽略了感知系统的行动导向作用。率失真理论定义了约束条件下的最优有损压缩,作为探索目标导向高效编码的形式化框架,已受到越来越多的关注。在本研究中,我们在率失真理论的语境下探索以行动为中心的表征。我们还给出了抽象(abstractions)的数学定义,并提出,作为相关细节的总结,抽象可用于确定以行动为中心的表征的内容。我们使用变分自编码器(VAE)对以行动为中心的表征进行建模,发现这类表征:i)是数据的高效有损压缩;ii)捕捉了实现成功行为所必需的、依赖于任务的不变性;iii)并非服务于数据重构。因此我们得出结论:实现最优行为几乎不需要完整的数据重构,这与感知的目的论方法相一致。
英文摘要:
Organisms have to keep track of the information in the environment that is relevant for adaptive behaviour. Transmitting information in an economical and efficient way becomes crucial for limited-resourced agents living in high-dimensional environments. The efficient coding hypothesis claims that organisms seek to maximize the information about the sensory input in an efficient manner. Under Bayesian inference, this means that the role of the brain is to efficiently allocate resources in order to make predictions about the hidden states that cause sensory data. However, neither of those frameworks accounts for how that information is exploited downstream, leaving aside the action-oriented role of the perceptual system. Rate-distortion theory, which defines optimal lossy compression under constraints, has gained attention as a formal framework to explore goal-oriented efficient coding. In this work, we explore action-centric representations in the context of rate-distortion theory. We also provide a mathematical definition of abstractions and we argue that, as a summary of the relevant details, they can be used to fix the content of action-centric representations. We model action-centric representations using VAEs and we find that such representations i) are efficient lossy compressions of the data; ii) capture the task-dependent invariances necessary to achieve successful behaviour; and iii) are not in service of reconstructing the data. Thus, we conclude that full reconstruction of the data is rarely needed to achieve optimal behaviour, consistent with a teleological approach to perception.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。