置信度和离散度说话：用于无监督精度估计的特征预测矩阵
Confidence and Dispersity Speak: Characterising Prediction Matrix for Unsupervised Accuracy Estimation
摘要：
这项工作的目的是评估如何以及一个模型下执行分布偏移不使用标签。虽然最近的方法研究预测的信心，这项工作报告预测分散性是另一个信息线索。置信度反映了单个预测是否确定;分散度表明总体预测如何在所有类别中分布。我们的关键见解是，一个表现良好的模型应该给出高置信度和高分散性的预测。也就是说，我们需要考虑这两个属性，以便做出更准确的估计。为此，我们使用的核规范，已被证明是有效的，在这两个属性的特征。大量的实验验证了核范数对各种模型的有效性（例如，ViT和ConvNeXt）、不同的数据集（例如，ImageNet和CUB-200）和各种类型的分布偏移（例如，风格转变和复制转变）。我们表明，核范数是更准确和更强大的精度估计比现有的方法。此外，我们还验证了其他测量方法的可行性（例如，互信息最大化）用于表征分散性和置信度。最后，我们研究了核范数的局限性，研究了其在严重类不平衡下的改进变体，并讨论了潜在的方向。
英文摘要：
This work aims to assess how well a model performs under distribution shifts without using labels. While recent methods study prediction confidence, this work reports prediction dispersity is another informative cue. Confidence reflects whether the individual prediction is certain; dispersity indicates how the overall predictions are distributed across all categories. Our key insight is that a well-performing model should give predictions with high confidence and high dispersity. That is, we need to consider both properties so as to make more accurate estimates. To this end, we use the nuclear norm that has been shown to be effective in characterizing both properties. Extensive experiments validate the effectiveness of nuclear norm for various models (e.g., ViT and ConvNeXt), different datasets (e.g., ImageNet and CUB-200), and diverse types of distribution shifts (e.g., style shift and reproduction shift). We show that the nuclear norm is more accurate and robust in accuracy estimation than existing methods. Furthermore, we validate the feasibility of other measurements (e.g., mutual information maximization) for characterizing dispersity and confidence. Lastly, we investigate the limitation of the nuclear norm, study its improved variant under severe class imbalance, and discuss potential directions.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
