减少两两比较的注释负担与人在环排序：在医学图像质量评估中的应用
Decreasing Annotation Burden of Pairwise Comparisons with Human-in-the-Loop Sorting: Application in Medical Image Artifact Rating
摘要：
通过成对比较进行的排名显示出比有序分类更高的可靠性。然而，由于成对比较的注释是二次缩放的，当数据集很大时，这变得不太实用。我们提出了一种方法，用于减少通过定量度量进行排名所需的成对比较的数量，证明了该方法在此概念研究证明中通过图像质量对医学图像进行排名的有效性。使用我们开发的医学图像注释软件，我们积极使用排序算法进行子样本成对比较，并在循环中加入人类评分员。我们发现，这种方法大大减少了一个完整的顺序排名所需的比较的数量，而不影响评分员之间的可靠性相比，成对比较，没有排序。
英文摘要：
Ranking by pairwise comparisons has shown improved reliability over ordinal classification. However, as the annotations of pairwise comparisons scale quadratically, this becomes less practical when the dataset is large. We propose a method for reducing the number of pairwise comparisons required to rank by a quantitative metric, demonstrating the effectiveness of the approach in ranking medical images by image quality in this proof of concept study. Using the medical image annotation software that we developed, we actively subsample pairwise comparisons using a sorting algorithm with a human rater in the loop. We find that this method substantially reduces the number of comparisons required for a full ordinal ranking without compromising inter-rater reliability when compared to pairwise comparisons without sorting.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
