arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~

从噪声中学习:受限玻尔兹曼机中的有效秩坍缩与分布外拒绝

Learning from Noise: Effective-Rank Collapse and Out-of-Distribution Rejection in Restricted Boltzmann Machines

Oshada Rathnayake, Nikhil Shukla

arXiv 2607.10506首次发表:更新:

发表机构

University of Virginia(弗吉尼亚大学)

机构由 AI 辅助整理,请以论文原文为准。

AI 中文总结

研究受限玻尔兹曼机在分布外输入下的脆弱性,通过分析可见-可见相互作用\(J\)的谱,提出用随机辅助曝光使相互作用有效秩坍缩,能在保持MNIST准确率时拒绝结构化OOD图像数据集。

AI 中文摘要

受限玻尔兹曼机(RBMs)通过在可见和隐藏配置上塑造能量景观来表示数据,但在分布外(OOD)输入下其判别性使用很脆弱。我们通过诱导的可见-可见相互作用\(J = WW^{T}\)的谱来分析这种失败模式。传统训练会将谱权重分散到许多与主体兼容的弱方向,增加\(J\)的有效秩。当在训练期间将辅助随机二值图像分配给拒绝标签时,学习到的相互作用会经历有效秩坍缩,\(J\)的有效秩接近经验数据协方差矩阵的秩。由此产生的RBM在保持MNIST分类准确率的同时拒绝结构化OOD图像数据集,表明随机辅助曝光可以重塑基于能量分类器的相互作用谱和自由能景观。

英文摘要

Restricted Boltzmann machines (RBMs) represent data by shaping an energy landscape over visible and hidden configurations, but their discriminative use is fragile under out-of-distribution (OOD) inputs: samples outside the training distribution can be absorbed into one of the learned class basins rather than rejected. Here, we analyze this failure mode through the spectrum of the induced visible--visible interaction $J=WW^{T}$, where \(W\) is the visible--hidden weight matrix. Relative to a Marchenko--Pastur random-matrix reference, conventional training spreads spectral weight into many weak, bulk-compatible directions, increasing the effective rank of $J$. When auxiliary random binary images are assigned to a rejection label during training, the learned interaction undergoes effective-rank collapse: weak bulk-like modes are depleted, spectral weight concentrates into fewer dominant eigendirections, and the effective rank of $J$ approaches that of the empirical data covariance matrix. The resulting RBM rejects structured OOD image datasets while preserving MNIST classification accuracy, showing that random auxiliary exposure can reshape both the interaction spectrum and the free-energy landscape of an energy-based classifier.

论文原文

arXiv 摘要页 · PDF 原文 · HTML 原文

↑