量化分类标注中的歧义性：一种度量与统计推断框架
Quantifying Ambiguity in Categorical Annotations: A Measure and Statistical Inference Framework
AI总结：
针对分类标注中软标签反映的歧义问题，提出一种非对称处理“无法解决”类别的歧义度量以分离两类不确定性，并开发了频率学派与贝叶斯统计推断工具，可用于数据集质量评估等场景。
摘要：
人工生成的分类标注通常会产生经验响应分布（软标签），这类分布反映的是标注歧义而非单纯的标注错误。我们提出一种歧义性度量方法，可将离散响应分布映射为单位区间内的标量，用于量化分类任务中的偶然不确定性。该度量与二次熵（基尼不纯度）密切相关，但区别于这类指标的是，它对显式的“无法解决”类别做非对称处理，从而将类别层面不可区分导致的不确定性与显式不可解性带来的不确定性分离开来。我们分析了该度量的形式化性质，并将其行为与文献中一种代表性的歧义度量进行了对比。除描述性分析外，我们还开发了用于推断的统计工具：提出了总体歧义性的频率学派点估计量，并推导了由底层概率向量上的Dirichlet先验诱导的歧义性贝叶斯后验，为认知不确定性提供了有理论依据的解释。数值示例展示了该方法在估计、校准，以及数据集质量评估和下游机器学习工作流中的实际应用。
英文摘要：
Human-generated categorical annotations frequently produce empirical response distributions (soft labels) that reflect ambiguity rather than simple annotator error. We introduce an ambiguity measure that maps a discrete response distribution to a scalar in the unit interval, designed to quantify aleatoric uncertainty in categorical tasks. The measure bears a close relationship to quadratic entropy (Gini-style impurity) but departs from those indices by treating an explicit "can't solve" category asymmetrically, thereby separating uncertainty arising from class-level indistinguishability from uncertainty due to explicit unresolvability. We analyze the measure's formal properties and contrast its behavior with a representative ambiguity measure from the literature. Moving beyond description, we develop statistical tools for inference: we propose frequentist point estimators for population ambiguity and derive the Bayesian posterior over ambiguity induced by Dirichlet priors on the underlying probability vector, providing a principled account of epistemic uncertainty. Numerical examples illustrate estimation, calibration, and practical use for dataset-quality assessment and downstream machine-learning workflows.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
