In psychological research often paired comparisons are used in which either full or partial profiles of the alternatives described by a common set of two-level attributes are presented. For this situation the problem of finding optimal designs is considered in the presence of third-order interactions.
It is shown how by not losing information on higher order interactions, optimal paired comparison designs involving alternatives of either full or partial profiles to reduce information overload as frequently encountered in applications can be constructed which enable identification of main effects up to third-order interactions when all attributes have general common number of levels.
For paired comparison experiments involving competing options described by two-level attributes several different methods of constructing designs having block paired observations under the main effects model are presented. These designs are compared to alternative designs available in the literature.
In paired comparison experiments respondents usually evaluate pairs of competing options. For this situation we introduce an appropriate model and derive optimal designs in the presence of second-order interactions when all attributes are dichotomous.
We present a Bayesian rating system based on the method of paired comparisons. Our system is a flexible generalization of the well-known Glicko, and in particular can better accommodate games with significant elements of luck. Our system is currently in use in the online game Duelyst II, and in that setting outperforms Glicko2.
A paired comparison analysis is the simplest way to make comparative judgments between objects where objects may be goods, services or skills. For a set of problems, this technique helps to choose the most important problem to solve first and/or provides the solution that will be the most effective. This paper presents the theory of paired comparisons method and contributes to the paired comparisons models by developing a new model based on t-distribution. The developed model is illustrated using a data set of citations among four famous journals of Statistics. Using Bayesian analysis, the jou…
In real life situations often paired comparisons involving alternatives of either full or partial profiles to mitigate cognitive burden are presented. For this situation the problem of finding optimal designs is considered in the presence of second-order interactions when all attributes have general common number of levels.
Thurstonian and Bradley-Terry models are the most commonly applied models in the analysis of paired comparison data. Since their introduction, numerous developments have been proposed in different areas. This paper provides an updated overview of these extensions, including how to account for object- and subject-specific covariates and how to deal with ordinal paired comparison data. Special emphasis is given to models for dependent comparisons. Although these models are more realistic, their use is complicated by numerical difficulties. We therefore concentrate on implementation issues. In pa…
Paired comparison models are used for analyzing data that involves pairwise comparisons among a set of objects. When the outcomes of the pairwise comparisons have no ties, the paired comparison models can be generalized as a class of binary response models. Receiver operating characteristic (ROC) curves and their corresponding areas under the curves are commonly used as performance metrics to evaluate the discriminating ability of binary response models. Despite their individual wide range of usage and their close connection to binary response models, ROC analysis to our knowledge has never be…
This article introduces the bpcs R package (Bayesian Paired Comparison in Stan) and the statistical models implemented in the package. This package aims to facilitate the use of Bayesian models for paired comparison data in behavioral research. Bayesian analysis of paired comparison data allows parameter estimation even in conditions where the maximum likelihood does not exist, allows easy extension of paired comparison models, provide straightforward interpretation of the results with credible intervals, have better control of type I error, have more robust evidence towards the null hypothesi…
In a wide class of paired comparisons, especially in the sports games, in which all subjects are divided into several groups, the intragroup comparisons are dense and the intergroup comparisons are sparse. Typical examples include the NFL regular season. Motivated by these situations, we propose group sparsity for paired comparisons and show the consistency and asymptotical normality of the maximum likelihood estimate in the Bradley-Terry model when the number of parameters goes to infinity in this paper. Simulations are carried out to illustrate the group sparsity and asymptotical results.
Scientists often use a paired comparison of the areas under the receiver operating characteristic curves to decide which continuous cancer screening test has the best diagnostic accuracy. In the paired design, all participants are screened with both tests. Participants with unremarkable screening results enter a follow-up period. Participants with suspicious screening results and those who show evidence of disease during follow-up receive the gold standard test. The remaining participants are classified as non-cases, even though some may have occult disease. The standard analysis includes all …
The authors give an approximation method for Bayesian inference in arena model, which is focused on paired comparisons with eliminations and bifurcations. The approximation method simplifies the inference by reducing parameters and introducing normal distribution functions into the computation of posterior distribution, which is largely based on an important property of normal random variables. Maximum a posteriori probability (MAP) and Bayesian prediction are then used to mine the information from the past pairwise comparison data, such as an individual's strength or volatility and his possib…