玻尔兹曼机的去学习正则化
Unlearning regularization for Boltzmann Machines
AI总结:
本文提出一种用于玻尔兹曼机的去学习正则化方法,以提高模型在参数重新缩放下的鲁棒性,避免临界性导致的过拟合和采样缓慢问题,并在合成数据与真实神经元数据上验证了其优于$L_p$-范数方案的有效性。
摘要:
玻尔兹曼机(BMs)是具有互连二值单元的图模型,用于数据分布的无监督建模。当在真实数据上训练时,BMs表现出像临界系统一样行为的倾向,显示模型在推断参数的小幅重新缩放下具有高敏感性。这种行为不便于生成数据的目的,因为它减慢了采样过程,并导致模型过拟合训练数据。在本研究中,我们引入了一种BMs的正则化方法,以提高模型在参数重新缩放下的鲁棒性。这项新技术与去学习算法在形式上具有相似性,后者是一种用于改进Hopfield类神经网络中记忆关联性的迭代过程。我们在由两个简单模型生成的合成数据上测试了我们的去学习正则化,即Curie-Weiss铁磁模型和Sherrington-Kirkpatrick自旋玻璃模型。我们表明它优于$L_p$-范数方案,并讨论了参数初始化的作用。最终,该方法被应用于学习真实神经元细胞的活动,证实了其在将推断模型从临界状态移开方面的有效性,并作为实际科学实现的有力候选者。
英文摘要:
Boltzmann Machines (BMs) are graphical models with interconnected binary units, employed for the unsupervised modeling of data distributions. When trained on real data, BMs show the tendency to behave like critical systems, displaying a high susceptibility of the model under a small rescaling of the inferred parameters. This behaviour is not convenient for the purpose of generating data, because it slows down the sampling process, and induces the model to overfit the training-data. In this study, we introduce a regularization method for BMs to improve the robustness of the model under rescaling of the parameters. The new technique shares formal similarities with the unlearning algorithm, an iterative procedure used to improve memory associativity in Hopfield-like neural networks. We test our unlearning regularization on synthetic data generated by two simple models, the Curie-Weiss ferromagnetic model and the Sherrington-Kirkpatrick spin glass model. We show that it outperforms $L_p$-norm schemes and discuss the role of parameter initialization. Eventually, the method is applied to learn the activity of real neuronal cells, confirming its efficacy at shifting the inferred model away from criticality and coming out as a powerful candidate for actual scientific implementations.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。