arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2501.15977cs.LGstat.ML
收藏

机器学习中低贝叶斯错误率条件下的分类错误率界限

Classification Error Bound for Low Bayes Error Conditions in Machine Learning

RWTH Aachen University(亚琛工业大学);AppTek GmbH(AppTek有限公司)

Zijian Yang, Vahe Eminyan, Ralf Schlüter, Hermann Ney

AI总结:

针对机器学习中模型分布替换导致的分类错误率不匹配问题,本文基于Kullback-Leibler散度提出低贝叶斯错误率条件下的线性近似界限,扩展至序列并应用于语音识别,分析了交叉熵损失、困惑度与词错误率的相关性。

摘要:

在统计分类和机器学习中,分类错误率是一项重要的性能度量,贝叶斯决策规则可使其最小化。在实践中,贝叶斯决策规则中未知的真实分布通常被替换为由训练数据估计出的模型分布。这种替换在贝叶斯错误率和基于模型的分类错误率之间引入了不匹配。在本研究中,我们应用分类错误率界限来研究机器学习中错误率不匹配与Kullback-Leibler散度之间的关系。受许多机器学习任务中观察到的基于模型的分类错误率较低这一现象的启发,我们将贝叶斯错误率界定得更低,针对低贝叶斯错误率条件,提出了一种分类错误率界限的线性近似。接着,讨论了类先验的界限。此外,我们将分类错误率界限扩展到了序列上。以自动语音识别作为机器学习应用的代表性示例,本研究分析性地讨论了在扩展界限下不同性能度量之间的相关性,包括交叉熵损失、语言模型困惑度和词错误率。

英文摘要:

In statistical classification and machine learning, classification error is an important performance measure, which is minimized by the Bayes decision rule. In practice, the unknown true distribution is usually replaced with a model distribution estimated from the training data in the Bayes decision rule. This substitution introduces a mismatch between the Bayes error and the model-based classification error. In this work, we apply classification error bounds to study the relationship between the error mismatch and the Kullback-Leibler divergence in machine learning. Motivated by recent observations of low model-based classification errors in many machine learning tasks, bounding the Bayes error to be lower, we propose a linear approximation of the classification error bound for low Bayes error conditions. Then, the bound for class priors are discussed. Moreover, we extend the classification error bound for sequences. Using automatic speech recognition as a representative example of machine learning applications, this work analytically discusses the correlations among different performance measures with extended bounds, including cross-entropy loss, language model perplexity, and word error rate.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑