关于机器学习中可扩展熵突破过拟合障碍的研究
On a scalable entropic breaching of the overfitting barrier in machine learning
摘要：
当相对小的数据统计大小$T$不足以为相对大的数据特征维度$D$提供鲁棒的ML拟合时，“小数据”的过拟合和处理是机器学习（ML）中最具挑战性的问题之一。通过对不同$D$和$T$的通用分类问题进行并行ML分析，证明了常见ML方法存在显著的线性过拟合障碍。例如，这些结果表明，对于使用长短期记忆深度学习分类器（LSTM）对生物信息学驱动的一般问题进行鲁棒分类，在最佳情况下，需要至少比特征维度$D$大13.8倍的统计数据$T$。结果表明，这种过拟合障碍可以突破$10^{-12}$分数的计算成本的熵最优的可扩展概率近似算法（eSPA）的装置，执行联合解决方案的熵最优贝叶斯网络推理和特征空间分割问题。与标准生物信息学工具相比，eSPA应用于实验性单细胞RNA测序数据的分类性能提高了30倍，与深度学习LSTM分类器相比提高了7倍。
英文摘要：
Overfitting and treatment of "small data" are among the most challenging problems in the machine learning (ML), when a relatively small data statistics size $T$ is not enough to provide a robust ML fit for a relatively large data feature dimension $D$. Deploying a massively-parallel ML analysis of generic classification problems for different $D$ and $T$, existence of statistically-significant linear overfitting barriers for common ML methods is demonstrated. For example, these results reveal that for a robust classification of bioinformatics-motivated generic problems with the Long Short-Term Memory deep learning classifier (LSTM) one needs in a best case a statistics $T$ that is at least 13.8 times larger then the feature dimension $D$. It is shown that this overfitting barrier can be breached at a $10^{-12}$ fraction of the computational cost by means of the entropy-optimal Scalable Probabilistic Approximations algorithm (eSPA), performing a joint solution of the entropy-optimal Bayesian network inference and feature space segmentation problems. Application of eSPA to experimental single cell RNA sequencing data exhibits a 30-fold classification performance boost when compared to standard bioinformatics tools - and a 7-fold boost when compared to the deep learning LSTM classifier.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
