Classification is a ubiquitous and fundamental problem in artificial intelligence and machine learning, with extensive efforts dedicated to developing more powerful classifiers and larger datasets. However, the classification task is ultimately constrained by the intrinsic properties of datasets, independently of computational power or model complexity. In this work, we introduce a formal entropy-based measure of classificability, which quantifies the inherent difficulty of a classification problem by assessing the uncertainty in class assignments given feature representations. This measure captures the degree of class overlap and aligns with human intuition, serving as an upper bound on classification performance for classification problems. Our results establish a theoretical limit beyond which no classifier can improve the classification accuracy, regardless of the architecture or amount of data, in a given problem. Our approach provides a principled framework for understanding when classification is inherently fallible and fundamentally ambiguous.
On support vector machines under a multiple-cost scenario
IMUS, Instituto de Matemáticas de la Universidad de Sevilla（塞维利亚大学数学研究所）；Departamento de Estadística e Investigación Operativa, Universidad de Sevilla（塞维利亚大学统计与运筹学系）；Departamento de Estadística e Investigación Operativa, Universidad de Cádiz（加的斯大学统计与运筹学系）
Werner Reichardt Centre for Integrative Neuroscience（维尔纳·赖卡特整合神经科学中心）；Bernstein Center for Computational Neuroscience（伯恩斯坦计算神经科学中心）；Max-Planck-Institute for Biological Cybernetics（马克斯·普朗克生物控制论研究所）
