最小化分类错误概率的最优压缩：一种信息论方法
Optimal Compression for Minimizing Classification Error Probability: an Information-Theoretic Approach
摘要：
我们制定的约束下，压缩数据可以用于机器学习中的准确分类进行最佳数据压缩的问题。我们表明，这转化为一个问题，最小化数据和压缩版本之间的互信息的约束下的分类错误概率是小的，当使用压缩数据进行机器学习。然后，我们提供了分析和计算方法来表征数据压缩和分类错误概率之间的最佳权衡。首先，我们提供了一个分析表征的最佳压缩策略的数据与二进制标签。其次，对于具有多个标签的数据，我们制定了一组凸优化问题来表征最佳权衡，通过数值求解制定的优化问题，可以获得分类错误和压缩效率之间的最佳权衡。我们进一步展示了我们的配方在分类性能的信息瓶颈方法的改进。
英文摘要：
We formulate the problem of performing optimal data compression under the constraints that compressed data can be used for accurate classification in machine learning. We show that this translates to a problem of minimizing the mutual information between data and its compressed version under the constraint on error probability of classification is small when using the compressed data for machine learning. We then provide analytical and computational methods to characterize the optimal trade-off between data compression and classification error probability. First, we provide an analytical characterization for the optimal compression strategy for data with binary labels. Second, for data with multiple labels, we formulate a set of convex optimization problems to characterize the optimal tradeoff, from which the optimal trade-off between the classification error and compression efficiency can be obtained by numerically solving the formulated optimization problems. We further show the improvements of our formulations over the information-bottleneck methods in classification performance.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
