掩码图像建模作为跨眼动自监督学习的框架
Masked Image Modeling as a Framework for Self-Supervised Learning across Eye Movements
AI总结：
本文以掩码图像建模为框架，将眼动与灵长类聚焦视觉建模为生成式自监督任务，揭示其潜在空间解耦及与不变性、潜在正则化学习的联系。
摘要：
为了理解周围环境，智能系统必须将复杂的感官输入转换为结构化编码，这些编码被缩减为与任务相关的信息，例如物体类别。生物智能体主要以自主方式实现这一点，据推测是通过自监督学习。尽管以往对其底层机制进行建模的尝试本质上主要是判别性的，但有充分证据表明大脑采用了关于世界的生成模型。在本文中，我们提出眼动与灵长类视觉的聚焦特性相结合，构成了一项预测并揭示视觉信息的生成式自监督任务。我们从掩码图像建模（MIM）这一深度表征学习中的常用方法出发，构建了一个原理验证模型。为此，我们分析了MIM的核心组件（如掩码技术和数据增强）如何影响类别特异性表征的形成。这不仅使我们能够更好地理解MIM背后的原理，还能随后重组出一个更符合生物感知聚焦特性的MIM。我们发现，MIM在没有显式正则化的情况下解耦了潜在空间中的神经元，这一特性被认为构成了灵长类动物视觉表征的结构。结合以往关于不变性学习的发现，这凸显了MIM与自监督学习的潜在正则化方法之间的有趣联系。源代码可在https://github.com/RobinWeiler/FocusMIM获取。
英文摘要：
To make sense of their surroundings, intelligent systems must transform complex sensory inputs to structured codes that are reduced to task-relevant information such as object category. Biological agents achieve this in a largely autonomous manner, presumably via self-supervised learning. Whereas previous attempts to model the underlying mechanisms were largely discriminative in nature, there is ample evidence that the brain employs a generative model of the world. Here, we propose that eye movements, in combination with the focused nature of primate vision, constitute a generative, self-supervised task of predicting and revealing visual information. We construct a proof-of-principle model starting from the framework of masked image modeling (MIM), a common approach in deep representation learning. To do so, we analyze how core components of MIM such as masking technique and data augmentation influence the formation of category-specific representations. This allows us not only to better understand the principles behind MIM, but to then reassemble a MIM more in line with the focused nature of biological perception. We find that MIM disentangles neurons in latent space without explicit regularization, a property that has been suggested to structure visual representations in primates. Together with previous findings of invariance learning, this highlights an interesting connection of MIM to latent regularization approaches for self-supervised learning. The source code is available under https://github.com/RobinWeiler/FocusMIM
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
