The standard linear and logistic regression models assume that the response variables are independent, but share the same linear relationship to their corresponding vectors of covariates. The assumption that the response variables are independent is, however, too strong. In many applications, these responses are collected on nodes of a network, or some spatial or temporal domain, and are dependent. Examples abound in financial and meteorological applications, and dependencies naturally arise in social networks through peer effects. Regression with dependent responses has thus received a lot of attention in the Statistics and Economics literature, but there are no strong consistency results unless multiple independent samples of the vectors of dependent responses can be collected from these models. We present computationally and statistically efficient methods for linear and logistic regression models when the response variables are dependent on a network. Given one sample from a networked linear or logistic regression model and under mild assumptions, we prove strong consistency results for recovering the vector of coefficients and the strength of the dependencies, recovering the rates of standard regression under independent observations. We use projected gradient descent on the negative log-likelihood, or negative log-pseudolikelihood, and establish their strong convexity and consistency using concentration of measure for dependent random variables.
Distribution-dependent Generalization Bounds for Tuning Linear Regression Across Tasks
School of Computer Science, Carnegie Mellon University（卡内基梅隆大学计算机科学学院）；Machine Learning Department, Carnegie Mellon University（卡内基梅隆大学机器学习系）；Toyota Technological Institute at Chicago（芝加哥丰田技术学院）；Northwestern University（西北大学）
Adaptive deep nonparametric regression from dependent data under covariate shift
Université Jean Monnet, ICJ UMR5208, CNRS, Ecole Centrale de Lyon, INSA Lyon, Universite Claude Bernard Lyon 1（里昂 Jean Monnet 大学，ICJ UMR5208，国家科学研究中心，里昂中央理工大学，里昂国家理工学校，里昂大学 Claude Bernard 1）
Technion — Israel Institute of Technology（以色列理工学院）
我们考虑线性回归的最常见变体，包括Ridge，Lasso和Support vector回归，在这种设置中，学习者在训练时只允许观察每个示例的固定数量的属性。我们提出了这些问题的简单而有效的算法：对于Lasso和Ridge回归，它们需要与全信息算法相同的属性总数（直到常数），以达到一定的精度。对于支持向量回归，与现有技术相比，我们需要指数级更少的属性。通过这一点，我们解决了Cesa-Bianchi et al.（2010）最近提出的一个开放问题。实验表明，与现有技术相比，优越的性能证明了理论界限的合理性。
Spectrum Dependent Learning Curves in Kernel Regression and Wide Neural Networks
John A. Paulson School of Engineering and Applied Sciences, Harvard University（哈佛大学约翰·A·保尔森工程与应用科学学院）；Center for Brain Science, Harvard University（哈佛大学脑科学中心）
Multivariate Regression with Grossly Corrupted Observations: A Robust Approach and its Applications
Bioinformatics Institute, A*STAR（新加坡科技研究局生物信息研究所）；Institute of Psychology, Chinese Academy of Sciences（中国科学院心理研究所）；School of Computing, National University of Singapore（新加坡国立大学计算学院）
Three-quarter Sibling Regression for Denoising Observational Data
University of Massachusetts Amherst（马萨诸塞大学阿默斯特分校）；Mount Holyoke College（曼荷莲学院）；Amazon（亚马逊）；University of Georgia（佐治亚大学）；Oregon State University（俄勒冈州立大学）
