神经粗粒化:用神经网络提取缓慢变化的潜在自由度
Neural Coarse-Graining: Extracting slowly-varying latent degrees of freedom with neural networks
摘要:
我们为神经网络提出了一个损失函数,它包含了平凡预测与非平凡预测的概念,这样网络就可以共同确定自己的预测目标并学会满足这些目标。这允许网络选择最适合其能力的问题子集,同时丢弃干扰其学习的“分散注意力”的元素。为此,网络首先将原始数据转换为更高级别的分类表示,然后从新的时间序列中训练预测器。为了防止将信号映射到零的平凡解决方案,我们通过学习模型的预测误差与整体信号统计的朴素模型之间的对比来引入非平凡性的度量。该变换可以学习丢弃信号的无信息和不可预测的分量,以支持高度预测和高度可预测的特征。这创建了时间序列动态的粗粒度模型,专注于预测控制时间序列统计的缓慢变化的潜在参数,而不是直接预测快速细节。其结果是一个半监督算法,它能够提取潜在参数,分割具有不同统计数据的时间序列部分,并从未标记的数据中构建底层动态的更高级别的表示。
英文摘要:
We present a loss function for neural networks that encompasses an idea of trivial versus non-trivial predictions, such that the network jointly determines its own prediction goals and learns to satisfy them. This permits the network to choose sub-sets of a problem which are most amenable to its abilities to focus on solving, while discarding 'distracting' elements that interfere with its learning. To do this, the network first transforms the raw data into a higher-level categorical representation, and then trains a predictor from that new time series to its future. To prevent a trivial solution of mapping the signal to zero, we introduce a measure of non-triviality via a contrast between the prediction error of the learned model with a naive model of the overall signal statistics. The transform can learn to discard uninformative and unpredictable components of the signal in favor of the features which are both highly predictive and highly predictable. This creates a coarse-grained model of the time-series dynamics, focusing on predicting the slowly varying latent parameters which control the statistics of the time-series, rather than predicting the fast details directly. The result is a semi-supervised algorithm which is capable of extracting latent parameters, segmenting sections of time-series with differing statistics, and building a higher-level representation of the underlying dynamics from unlabeled data.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。