Concentration of Benefit index: A threshold-free summary metric for quantifying the capacity of covariates to yield efficient treatment rules
Mohsen Sadatsafavi, Mohammad Ali Mansournia, Paul Gustafson
备注：
This submission was intended to be an update of the previous work (arXiv:1901.05124) and not a new record. As such, the authors decided to withdraw this record and will update arXiv:1901.05124 with an identical copy
When data on treatment assignment, outcomes, and covariates from a randomized trial are available, a question of interest is to what extent covariates can be used to optimize treatment decisions. Statistical hypothesis testing of covariate-by-treatment interaction is ill-suited for this purpose. The application of decision theory results in treatment rules that compare the expected benefit of treatment given the patient's covariates against a treatment threshold. However, determining treatment threshold is often context-specific, and any given threshold might seem arbitrary when the overall capacity towards predicting treatment benefit is of concern. We propose the Concentration of Benefit index (Cb), a threshold-free metric that quantifies the combined performance of covariates towards finding individuals who will benefit the most from treatment. The construct of the proposed index is comparing expected treatment outcomes with and without knowledge of covariates when one of a two randomly selected patients are to be treated. We show that the resulting index can also be expressed in terms of the integrated efficiency of individualized treatment decision over the entire range of treatment thresholds. We propose parametric and semi-parametric estimators, the latter being suitable for out-of-sample validation and correction for optimism. We used data from a clinical trial to demonstrate the calculations in a step-by-step fashion, and have provided the R code for implementation (https://github.com/msadatsafavi/txBenefit). The proposed index has intuitive and theoretically sound interpretation and can be estimated with relative ease for a wide class of regression models. Beyond the conceptual developments, various aspects of estimation and inference for such a metric need to be pursued in future research.
Transportable inference using target population summary statistics under covariate shift
Academy of Mathematics and Systems Science Chinese Academy of Sciences（中国科学院数学与系统科学研究院）；Department of Biostatistics Mailman School of Public Health Columbia University（哥伦比亚大学麦尔曼公共卫生学院生物统计学系）；Department of Epidemiology & Biostatistics University of California at San Francisco（旧金山大学生物统计与流行病学系）
The focus of this paper is on quantifying the capacity of covariates in devising efficient treatment rules when data from a randomized trial are available. Conventional one-variable-at-a-time subgroup analysis based on statistical hypothesis testing of covariate-by-treatment interaction is ill-suited for this purpose. The application of decision theory results in treatment rules that compare the expected benefit of treatment given the patient's covariates against a treatment threshold. However, determining treatment threshold is often context-specific, and any given threshold might seem arbitr…
An important problem in network analysis is predicting a node attribute using both network covariates, such as graph embedding coordinates or local subgraph counts, and conventional node covariates, such as demographic characteristics. While standard regression methods that make use of both types of covariates may be used for prediction, statistical inference is complicated by the fact that the nodal summary statistics are often dependent in complex ways. We show that under a mild joint exchangeability assumption, a network analog of conformal prediction achieves finite sample validity for a w…
We propose a method for summarizing the strength of association between a set of variables and a multivariate outcome. Classical summary measures are appropriate when linear relationships exist between covariates and outcomes, while our approach provides an alternative that is useful in situations where complex relationships may be present. We utilize ensemble machine learning to detect nonlinear relationships and covariate interactions and propose a measure of association that captures these relationships. A hypothesis test about the proposed associative measure can be used to test the strong…
Individualizing treatment assignment can improve outcomes for diseases with patient-to-patient variability in comparative treatment effects. When a clinical trial demonstrates that some patients improve on treatment while others do not, it is tempting to assume that treatment effect heterogeneity exists. However, if variability in response is mainly driven by factors other than treatment, investigating the extent to which covariate data can predict differential treatment response is a potential waste of resources. Motivated by recent meta-analyses assessing the potential of individualizing tre…
Receiver operating characteristic (ROC) analysis is one of the most popular approaches for evaluating and comparing the accuracy of medical diagnostic tests. Although various methodologies have been developed for estimating ROC curves and its associated summary indices, there is no consensus on a single framework that can provide consistent statistical inference whilst handling the complexities associated with medical data. Such complexities might include covariates that influence the diagnostic potential of a test, ordinal test data, censored data due to instrument detection limits or correla…
Precise High-Dimensional Asymptotics for Quantifying Heterogeneous Transfers
Yau Mathematical Sciences Center, Tsinghua University（清华大学数学科学中心）；Khoury College of Computer Sciences, Northeastern University（东北大学计算机科学学院）；Department of Computer Science, Stanford University（斯坦福大学计算机科学系）；Department of Statistics and Data Science, University of Pennsylvania（宾夕法尼亚大学统计与数据科学系）
Measuring performance & quantifying a performance change are core evaluation techniques in programming language and systems research. Of 122 recent scientific papers, as many as 65 included experimental evaluation that quantified a performance change using a ratio of execution times. Few of these papers evaluated their results with the level of rigour that has come to be expected in other experimental sciences. The uncertainty of measured results was largely ignored. Scarcely any of the papers mentioned uncertainty in the ratio of the mean execution times, and most did not even mention uncerta…
While it is common practice in applied network analysis to report various standard network summary statistics, these numbers are rarely accompanied by uncertainty quantification. Yet any error inherent in the measurements underlying the construction of the network, or in the network construction procedure itself, necessarily must propagate to any summary statistics reported. Here we study the problem of estimating the density of an arbitrary subgraph, given a noisy version of some underlying network as data. Under a simple model of network error, we show that consistent estimation of such dens…
The paper considers the problem of multi-objective decision support when outcomes are uncertain. We extend the concept of Pareto-efficient decisions to take into account the uncertainty of decision outcomes across varying contexts. This enables quantifying trade-offs between decisions in terms of tail outcomes that are relevant in safety-critical applications. We propose a method for learning efficient decisions with statistical confidence, building on results from the conformal prediction literature. The method adapts to weak or nonexistent context covariate overlap and its statistical guaran…
Inferring resource selection and utilization distributions from irregular and error-prone animal tracking data
University of British Columbia（不列颠哥伦比亚大学）；NOAA-NMFS Alaska Fisheries Science Center（美国国家海洋和大气管理局鱼类渔业科学中心阿拉斯加分中心）；University of Washington（华盛顿大学）；Bielefeld University（比勒费尔德大学）；Fisheries and Oceans Canada（加拿大渔业与海洋部）；University of Windsor（温莎大学）
HIV dynamic studies have contributed significantly to the understanding of HIV pathogenesis and antiviral treatment strategies for AIDS patients. Establishing the relationship of virologic responses with clinical factors and covariates during long-term antiretroviral (ARV) therapy is important to the development of effective treatments. Medication adherence is an important predictor of the effectiveness of ARV treatment, but an appropriate determinant of adherence rate based on medication event monitoring system (MEMS) data is critical to predict virologic outcomes. The primary objective of th…
How Fast Do Signatures Learn? Statistical Theory and Applications for Path Regression
Mathematical Institute, University of Oxford（牛津大学数学研究所）；Center for Data Science, Peking University（北京大学数据科学中心）；School of Mathematical Sciences, Peking University（北京大学数学科学学院）；Oxford Man Institute, University of Oxford（牛津大学牛津人学研究所）；Center for Statistical Science, Peking University（北京大学统计科学中心）
We introduce the transport-transform (TT) and the relative transport-transform (RTT) metrics between finite point patterns on a general space, which provide a unified framework for earlier point pattern metrics, in particular the generalized spike time and the normalized and unnormalized OSPA metrics. Our main focus is on barycenters, i.e. minimizers of a $q$-th order Fréchet functional with respect to these metrics. We present a heuristic algorithm that terminates in a local minimum and is shown to be fast and reliable in a simulation study. The algorithm serves as an umbrella method that can…
The rates of respiratory prescriptions vary by GP surgery across Scotland, suggesting there are sizeable health inequalities in respiratory ill health across the country. The aim of this paper is to estimate the magnitude, spatial pattern and drivers of this spatial variation. Monthly data on respiratory prescriptions are available at the GP surgery level, which creates an interesting methodological challenge as these data are not the classical geostatistical, areal unit or point process data types. A novel process-convolution model is proposed, which extends existing methods by being an adapt…
The difference in restricted mean survival times between two groups is a clinically relevant summary measure. With observational data, there may be imbalances in confounding variables between the two groups. One approach to account for such imbalances is to estimate a covariate-adjusted restricted mean difference by modeling the covariate-adjusted survival distribution and then marginalizing over the covariate distribution. We demonstrate that the mean-squared error of the restricted mean difference is bounded by the mean-squared error of the covariate-adjusted survival distribution estimators…
Personalized decision-making, aiming to derive optimal treatment regimes based on individual characteristics, has recently attracted increasing attention in many fields, such as medicine, social services, and economics. Current literature mainly focuses on estimating treatment regimes from a single source population. In real-world applications, the distribution of a target population can be different from that of the source population. Therefore, treatment regimes learned by existing methods may not generalize well to the target population. Due to privacy concerns and other practical issues, i…
A neighbour selection approach for identifying differential networks in conditional functional graphical models
Politecnico di Milano（米兰理工大学）；Human Technopole（人类技术顶石研究所）；IRCCS Azienda Ospedaliera Metropolitana（热那亚都市医院IRCCS）；Università degli Studi di Genova（热那亚大学）
Models for dependent data are distinguished by their targets of inference. Marginal models are useful when interest lies in quantifying associations averaged across a population of clusters. When the functional form of a covariate-outcome association is unknown, flexible regression methods are needed to allow for potentially non-linear relationships. We propose a novel marginal additive model (MAM) for modelling cluster-correlated data with non-linear population-averaged associations. The proposed MAM is a unified framework for estimation and uncertainty quantification of a marginal mean model…
Estimating the impact of trauma treatment protocols is complicated by the high dimensional yet finite sample nature of trauma data collected from observational studies. Viscoelastic assays are highly predictive measures of hemostasis. However, the effectiveness of thromboelastography(TEG) based treatment protocols has not been statistically evaluated.To conduct robust and reliable estimation with sparse data, we built an estimation "machine" for estimating causal impacts of candidate variables using the collaborative targeted maximum loss-based estimation(CTMLE) framework.The computational eff…
