arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 1605.01749cs.LGstat.ML
收藏

排序自动编码器

Rank Ordered Autoencoders

Paul Bertens

摘要:

提出并实现了一种利用自编码器对稀疏表示进行无监督学习的新方法,该方法将隐层单元的输出按其激活值排序,并按此顺序逐步重构输入。这可以与使用累积和和排序并行有效地完成,仅略微增加计算成本。最小化该渐进重构相对于输入的差异可以被看作是最小化重构输入所需的活动输出单元的数量。因此,模型学习使用最少数量的活动输出单元进行最佳重建。这导致了高稀疏性,而不需要额外的超参数,稀疏性的量是通过最小化这种渐进的重建误差来隐式学习的。针对CIFAR10数据集的补丁给出了训练模型的结果,显示了特征的快速收敛和极其稀疏的输出激活,同时保持了最小的重建误差,并显示了对过拟合的极端鲁棒性。此外,呈现了作为活动单元的数量的函数的重构,其示出了自动编码器在输入上学习等级顺序码,其中最高等级的单元对应于重构误差的最高降低。

英文摘要:

A new method for the unsupervised learning of sparse representations using autoencoders is proposed and implemented by ordering the output of the hidden units by their activation value and progressively reconstructing the input in this order. This can be done efficiently in parallel with the use of cumulative sums and sorting only slightly increasing the computational costs. Minimizing the difference of this progressive reconstruction with respect to the input can be seen as minimizing the number of active output units required for the reconstruction of the input. The model thus learns to reconstruct optimally using the least number of active output units. This leads to high sparsity without the need for extra hyperparameters, the amount of sparsity is instead implicitly learned by minimizing this progressive reconstruction error. Results of the trained model are given for patches of the CIFAR10 dataset, showing rapid convergence of features and extremely sparse output activations while maintaining a minimal reconstruction error and showing extreme robustness to overfitting. Additionally the reconstruction as function of number of active units is presented which shows the autoencoder learns a rank order code over the input where the highest ranked units correspond to the highest decrease in reconstruction error.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑