基于噪声性能数据的并列排序
Ranking with Ties based on Noisy Performance Data
AI总结:
本文研究在性能测量存在噪声时如何对对象进行并列排序,针对范围不可比关系不具传递性导致的排序冲突,定义了偏序排序,并提出三种计算方法以分析对象间的性能差异。
摘要:
我们考虑在性能测量受噪声影响时,基于性能对一组对象进行排序的问题。在此场景下,性能被重复测量,从而为每个对象产生一个测量范围。如果两个对象的范围不重叠,我们则认为一个对象比另一个“更好”,并期望它获得更高的排名;然而,如果范围重叠,则对象不可比,我们希望为其分配相同的排名。遗憾的是,范围的不可比关系通常不具有传递性;因此,这两个要求通常无法同时满足,即无法保证为范围分离的对象提供不同排名,同时为范围重叠的对象提供相同排名。这种冲突导致对一组对象进行排序存在不止一种合理方式。在本文中,我们探讨了并列排序时出现的歧义,并定义了一组合理的排序,我们称之为偏序排序。我们开发并分析了三种不同的方法来计算偏序排序。最后,我们展示了如何借助偏序排序来研究对象间的性能差异。
英文摘要:
We consider the problem of ranking a set of objects based on their performance when the measurement of said performance is subject to noise. In this scenario, the performance is measured repeatedly, resulting in a range of measurements for each object. If the ranges of two objects do not overlap, then we consider one object as 'better' than the other, and we expect it to receive a higher rank; if, however, the ranges overlap, then the objects are incomparable, and we wish them to be assigned the same rank. Unfortunately, the incomparability relation of ranges is in general not transitive; as a consequence, in general the two requirements cannot be satisfied simultaneously, i.e., it is not possible to guarantee both distinct ranks for objects with separated ranges, and same rank for objects with overlapping ranges. This conflict leads to more than one reasonable way to rank a set of objects. In this paper, we explore the ambiguities that arise when ranking with ties, and define a set of reasonable rankings, which we call partial rankings. We develop and analyse three different methodologies to compute a partial ranking. Finally, we show how performance differences among objects can be investigated with the help of partial ranking.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。