站在巨人的肩膀上
Standing on the shoulders of giants
AI总结：
本文提出用 IRT 心理测量信息扩展混淆矩阵，以补充精确率、F1 等经典指标，并在性能相近的模型间依据实例潜在特征进行更细粒度选择。
摘要：
尽管从混淆矩阵中提取的经典评估指标（如精确率和 F1）对机器学习的发展至关重要，但它们存在局限性。这类指标仅提供模型性能的定量视图，没有考虑数据的复杂性或命中结果的质量。为克服这些局限，近期研究引入了项目反应理论（Item Response Theory，IRT）等心理测量学指标，从而能够在实例的潜在特征层面进行评估。本研究考察 IRT 概念如何丰富混淆矩阵，以便在性能相近的多个候选模型中识别最合适的模型。在所开展的研究中，IRT 并非取代经典指标，而是对其加以补充，提供一个新的评估层，并观察模型在特定实例上的细微行为。研究还观察到，有 97% 的置信度认为，IRT 所得分数与所分析的 66% 的经典指标具有不同的贡献。
英文摘要：
Although fundamental to the advancement of Machine Learning, the classic evaluation metrics extracted from the confusion matrix, such as precision and F1, are limited. Such metrics only offer a quantitative view of the models' performance, without considering the complexity of the data or the quality of the hit. To overcome these limitations, recent research has introduced the use of psychometric metrics such as Item Response Theory (IRT), which allows an assessment at the level of latent characteristics of instances. This work investigates how IRT concepts can enrich a confusion matrix in order to identify which model is the most appropriate among options with similar performance. In the study carried out, IRT does not replace, but complements classical metrics by offering a new layer of evaluation and observation of the fine behavior of models in specific instances. It was also observed that there is 97% confidence that the score from the IRT has different contributions from 66% of the classical metrics analyzed.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
