Databases in domains such as healthcare are routinely released to the public in aggregated form. Unfortunately, naive modeling with aggregated data may significantly diminish the accuracy of inferences at the individual level. This paper addresses the scenario where features are provided at the individual level, but the target variables are only available as histogram aggregates or order statistics. We consider a limiting case of generalized linear modeling when the target variables are only known up to permutation, and explore how this relates to permutation testing; a standard technique for assessing statistical dependency. Based on this relationship, we propose a simple algorithm to estimate the model parameters and individual level inferences via alternating imputation and standard generalized linear model fitting. Our results suggest the effectiveness of the proposed approach when, in the original data, permutation testing accurately ascertains the veracity of the linear relationship. The framework is extended to general histogram data with larger bins - with order statistics such as the median as a limiting case. Our experimental results on simulated data and aggregated healthcare data suggest a diminishing returns property with respect to the granularity of the histogram - when a linear relationship holds in the original data, the targets can be predicted accurately given relatively coarse histograms.
Communication-Efficient Distributed Estimator for Generalized Linear Models with a Diverging Number of Covariates
School of Applied Sciences, Beijing Information Science and Technology University（北京信息科技大学应用科学学院）；School of Statistics, Renmin University of China（中国人民大学统计学院）；School of Statistics and Mathematics, Central University of Finance and Economics（中央财经大学统计与数学学院）
Efficient Aggregated Kernel Tests using Incomplete $U$-statistics
Centre for Artificial Intelligence（人工智能中心）；Gatsby Computational Neuroscience Unit（盖茨比计算神经科学研究组）；University College London（伦敦大学学院）；Inria London（法国国家信息与自动化研究所伦敦分部）；Yonsei University（延世大学）
A parsimonious tail compliant multiscale statistical model for aggregated rainfall
Univ Brest（布雷斯特大学）；CNRS UMR 6205, Laboratoire de Mathématiques de Bretagne Atlantique（法国国家科学研究中心 UMR 6205，布列塔尼大西洋数学实验室）；Dipartimento di Scienze Ambientali, Informatica e Statistica, Universita Ca’ Foscari di Venezia（威尼斯卡福斯卡里大学环境、信息学与统计学系）；Laboratoire des Sciences du Climat et l’Environnement（气候与环境科学实验室）
