深度网络中多感觉整合的关键学习期
Critical Learning Periods for Multisensory Integration in Deep Networks
AI总结：
本文发现神经网络在训练早期接触相关信号对多源信息整合至关重要，关键学习期源于复杂的早期瞬态动力学，并提出跨源重建辅助训练目标能显著增强网络对关键期的适应力。
摘要：
我们表明，神经网络整合来自不同来源信息的能力，关键取决于在训练早期阶段是否接触到适当相关的信号。在人工和生物系统中，干扰这一初始阶段的学习过程可能会永久损害技能的发展，这种现象被称为关键学习期。我们表明，关键期源于复杂且不稳定的早期瞬态动力学，这些动力学决定了受训系统的最终性能及其学习到的表征。这一证据挑战了一种观点，该观点由对宽且浅的网络的分析所引发，即神经网络的早期学习动力学很简单，类似于线性模型的动力学。事实上，我们表明即使是深层线性网络在多源整合方面也表现出关键学习期，而浅层网络则没有。为了更好地理解内部表征如何根据干扰或感觉缺陷发生变化，我们引入了一种新的源敏感性度量，使我们能够在训练过程中追踪源的抑制和整合。我们对抑制的分析表明，跨源重建是一种自然的辅助训练目标，事实上我们表明，使用跨传感器重建目标训练的架构对关键期具有更强的适应力。我们的发现表明，与以往有监督的努力相比，最近在自监督多模态训练中的成功，部分原因可能在于更稳健的学习动力学，而不仅仅是更好的架构和/或更多的数据。
英文摘要：
We show that the ability of a neural network to integrate information from diverse sources hinges critically on being exposed to properly correlated signals during the early phases of training. Interfering with the learning process during this initial stage can permanently impair the development of a skill, both in artificial and biological systems where the phenomenon is known as a critical learning period. We show that critical periods arise from the complex and unstable early transient dynamics, which are decisive of final performance of the trained system and their learned representations. This evidence challenges the view, engendered by analysis of wide and shallow networks, that early learning dynamics of neural networks are simple, akin to those of a linear model. Indeed, we show that even deep linear networks exhibit critical learning periods for multi-source integration, while shallow networks do not. To better understand how the internal representations change according to disturbances or sensory deficits, we introduce a new measure of source sensitivity, which allows us to track the inhibition and integration of sources during training. Our analysis of inhibition suggests cross-source reconstruction as a natural auxiliary training objective, and indeed we show that architectures trained with cross-sensor reconstruction objectives are remarkably more resilient to critical periods. Our findings suggest that the recent success in self-supervised multi-modal training compared to previous supervised efforts may be in part due to more robust learning dynamics and not solely due to better architectures and/or more data.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
