Calibration is a vital aspect of the performance of risk prediction models, but research in the context of ordinal outcomes is scarce. This study compared calibration measures for risk models predicting a discrete ordinal outcome, and investigated the impact of the proportional odds assumption on calibration and overfitting. We studied the multinomial, cumulative, adjacent category, continuation ratio, and stereotype logit/logistic models. To assess calibration, we investigated calibration intercepts and slopes, calibration plots, and the estimated calibration index. Using large sample simulations, we studied the performance of models for risk estimation under various conditions, assuming that the true model has either a multinomial logistic form or a cumulative logit proportional odds form. Small sample simulations were used to compare the tendency for overfitting between models. As a case study, we developed models to diagnose the degree of coronary artery disease (five categories) in symptomatic patients. When the true model was multinomial logistic, proportional odds models often yielded poor risk estimates, with calibration slopes deviating considerably from unity even on large model development datasets. The stereotype logistic model improved the calibration slope, but still provided biased risk estimates for individual patients. When the true model had a cumulative logit proportional odds form, multinomial logistic regression provided biased risk estimates, although these biases were modest. Non-proportional odds models require more parameters to be estimated from the data, and hence suffered more from overfitting. Despite larger sample size requirements, we generally recommend multinomial logistic regression for risk prediction modeling of discrete ordinal outcomes.
The R package multgee implements the local odds ratios generalized estimating equations (GEE) approach proposed by Touloumis et al. (2013), a GEE approach for correlated multinomial responses that circumvents theoretical and practical limitations of the GEE method. A main strength of multgee is that it provides GEE routines for both ordinal (ordLORgee) and nominal (nomLORgee) responses, while relevant softwares in R and SAS are restricted to ordinal responses under a marginal cumulative link model specification. In addition, multgee offers a marginal adjacent categories logit model for ordinal…
Generalized Odds for Ordinal Outcomes: A Causal Inference Framework Adjusting for Covariates
Beijing Institute of Brain Disorders, Capital Medical University（首都医科大学北京脑重大疾病研究院）；Chinese Institutes for Medical Research（中国医学科学院）；Department of Neurology, Xuanwu Hospital, Capital Medical University（首都医科大学宣武医院神经内科）
It is often desired that ordinal regression models yield unimodal predictions. However, in many recent works this characteristic is either absent, or implemented using soft targets, which do not guarantee unimodal outputs at inference. In addition, we argue that the standard maximum likelihood objective is not suitable for ordinal regression problems, and that optimal transport is better suited for this task, as it naturally captures the order of the classes. In this work, we propose a framework for deep ordinal regression, based on unimodal output distribution and optimal transport loss. Insp…
The ordinal endpoint is prevalent in clinical studies. For example, for the COVID-19, the most common endpoint used was 7-point ordinal scales. Another example is in phase II cancer studies, efficacy is often assessed as an ordinal variable based on a level of response of solid tumors with four categories: complete response, partial response, stable disease, and progression, though often a dichotomized approach is used in practices. However, there lack of designs for the ordinal endpoint despite Whitehead et al. (1993, 2017), Jaki et al. (2003) to list a few. In this paper, we propose a generi…
The potential of location-shift models to find adequate models between the proportional odds model and the non-proportional odds model is investigated. It is demonstrated that these models are very useful in ordinal modeling. While proportional odds models are often too simple, non proportional odds models are typically unnecessary complicated and seem widely dispensable. The class of location-shift models is also extended to allow for smooth effects. The additive location-shift model contains two functions for each explanatory variable, one for the location and one for dispersion. It is much …
In many randomized clinical trials of therapeutics for COVID-19, the primary outcome is an ordinal categorical variable, and interest focuses on the odds ratio (active agent vs. control) under the assumption of a proportional odds model. Although at the final analysis the outcome will be determined for all subjects, at an interim analysis, the status of some participants may not yet be determined, e.g., because ascertainment of the outcome may not be possible until some pre-specified follow-up time. Accordingly, the outcome from these subjects can be viewed as censored. A valid interim analysi…
Three statistical studies, all published between 2004 and 2008 but without referring to one another, assert a useful equivalence involving the hazard ratio, a parameter estimated for time to event data by the frequently used proportional hazards model. Stated generally, when the assumption of proportional hazards is justified, the hazard ratio is equivalent to the odds that a randomly chosen person from the group whose hazard is represented in the ratio's numerator will experience the event of interest before a randomly chosen person from the group represented by the ratio's denominator.
Software development innovations and advances in computing have enabled more complex and less costly computations in medical research (survival analysis), engineering studies (reliability analysis), and social sciences event analysis (historical analysis). As a result, many semi-parametric modeling efforts emerged when it comes to time-to-event data analysis. In this context, this work presents a flexible Bernstein polynomial (BP) based framework for survival data modeling. This innovative approach is applied to existing families of models such as proportional hazards (PH), proportional odds (…
In the statistical literature, a number of methods have been proposed to ensure valid inference about marginal effects of variables on a longitudinal outcome in settings with irregular monitoring times. However, the potential biases due to covariate-driven monitoring times and confounding have rarely been considered simultaneously, and never in a setting with an ordinal outcome and a continuous exposure. In this work, we propose and demonstrate a methodology for causal inference in such a setting, relying on a proportional odds model to study the effect of the exposure on the outcome. Irregula…
Evaluating the effect of different non-informative prior specifications on the Bayesian proportional odds model in randomised controlled trials: a simulation study
Sequential multiple assignment randomized trials (SMARTs) are used to construct data-driven optimal intervention strategies for subjects based on their intervention and covariate histories in different branches of health and behavioral sciences where a sequence of interventions is given to a participant. Sequential intervention strategies are often called dynamic treatment regimes (DTR). In the existing literature, the majority of the analysis methodologies for SMART data assume a continuous primary outcome. However, ordinal outcomes are also quite common in clinical practice. In this work, fi…
Multiple imputation (MI) is an established technique to handle missing data in observational studies. Joint modeling (JM) and fully conditional specification (FCS) are commonly used methods for imputing multilevel clustered data. However, MI approaches for ordinal clustered outcome variables have not been well studied, especially when there is informative cluster size (ICS). The purpose of this study is to describe different imputation and analysis strategies for the multilevel ordinal outcome when ICS exists. We conducted comprehensive Monte Carlo simulation studies to compare five different …
The use of the proportional odds (PO) model for ordinal regression is ubiquitous in the literature. If the assumption of parallel lines does not hold for the data, then an alternative is to specify a non-proportional odds (NPO) model, where the regression parameters are allowed to vary depending on the level of the response. However, it is often difficult to fit these models, and challenges regarding model choice and fitting are further compounded if there are a large number of explanatory variables. We make two contributions towards tackling these issues: firstly, we develop a Bayesian method…
We propose a novel Bayesian model framework for discrete ordinal and count data based on conditional transformations of the responses. The conditional transformation function is estimated from the data in conjunction with an a priori chosen reference distribution. For count responses, the resulting transformation model is novel in the sense that it is a Bayesian fully parametric yet distribution-free approach that can additionally account for excess zeros with additive transformation function specifications. For ordinal categoric responses, our cumulative link transformation model allows the i…
Multiple and combined endpoints involving also non-normal outcomes appear in many clinical trials in various areas in medicine. In some cases, the outcome can be observed only on an ordinal or dichotomous scale. Then the success of two therapies is assessed by comparing the outcome of two randomly selected patients from the two therapy groups by 'better', 'equal' or 'worse'. These outcomes can be described by the probabilities $p^-=P(X<Y)$, $p_0=P(X=Y)$, and $p^+ =P(X~>~Y)$. For a clinician, however, these quantities are less intuitive. Therefore, Noether (1987) introduced the quantity $λ=p^+ …
Ordinal cumulative probability models (CPMs) -- also known as cumulative link models -- such as the proportional odds regression model are typically used for discrete ordered outcomes, but can accommodate both continuous and mixed discrete/continuous outcomes since these are also ordered. Recent papers describe ordinal CPMs in this setting using non-parametric maximum likelihood estimation. We formulate a Bayesian CPM for continuous or mixed outcome data. Bayesian CPMs inherit many of the benefits of frequentist CPMs and have advantages with regard to interpretation, flexibility, and exact inf…
Bivariate ordered logistic models (BOLMs) are appealing to jointly model the marginal distribution of two ordered responses and their association, given a set of covariates. When the number of categories of the responses increases, the number of global odds ratios (or their re-parametrizations) to be estimated also increases and estimating the association structure becomes crucial for this type of data. In fact, such data could be too "rich" to be fully modelled with an ordinary BOLM while, sometimes, the well-known Dale's model could be too parsimonious to provide a good fit. In addition, whe…
Biplot Methods allow for the simultaneous representation of individuals and variables of a data matrix. For Binary or Nominal data, Logistic biplots have been recently developed to extend the classical linear representations for continuous data. When data are ordinal, linear, binary or nominal logistic biplots are not adequate and techniques as Categorical Principal Component Analysis (CATPCA) or Item Response Theory (IRT) for ordinal items should be used instead. In this paper we extend the Biplot to ordinal data. The resulting method is termed Ordinal Logistic Biplot (OLB). Row scores are co…
The proportional hazards (PH), proportional odds (PO) and accelerated failure time (AFT) models have been widely used in different applications of survival analysis. Despite their popularity, these models are not suitable to handle lifetime data with crossing survival curves. In 2005, Yang and Prentice proposed a semiparametric two-sample strategy (YP model), including the PH and PO frameworks as particular cases, to deal with this type of data. Assuming a general regression setting, the present paper proposes an unified approach to fit the YP model by employing Bernstein polynomials to manage…
