针对高维多任务学习中的 Data Enrichment/Sharing 问题,提出一种由凸函数刻画结构的公共与个体参数估计器,给出了非渐近误差界并设计了具有几何收敛率的迭代算法,首次提供了全面的统计与计算分析。
摘要:
我们考虑高维环境下的多任务学习问题。具体而言,我们引入了一个估计器,并研究了其在被称为 Data Enrichment/Sharing 的多重连通线性回归问题中的统计与计算性质。任务间的联系通过跨任务的“公共参数”来捕捉,并由各任务的“个体参数”进行细化。任何凸函数(如范数)均可刻画公共参数与个体参数的结构。我们刻画了该估计器的样本复杂度,并在几何条件下为所有参数的估计误差提供了高概率的非渐近界。我们证明,公共参数的恢复得益于“所有”汇聚样本。我们提出了一种具有几何收敛率的迭代估计算法,并通过合成数据上的实验补充了理论分析。总体而言,我们首次对数据共享模型中的推断进行了全面的统计与计算分析。
英文摘要:
We consider the problem of multi-task learning in the high dimensional setting. In particular, we introduce an estimator and investigate its statistical and computational properties for the problem of multiple connected linear regressions known as Data Enrichment/Sharing. The between-tasks connections are captured by a cross-tasks \emph{common parameter}, which gets refined by per-task \emph{individual parameters}. Any convex function, e.g., norm, can characterize the structure of both common and individual parameters. We delineate the sample complexity of our estimator and provide a high probability non-asymptotic bound for estimation error of all parameters under a geometric condition. We show that the recovery of the common parameter benefits from \emph{all} of the pooled samples. We propose an iterative estimation algorithm with a geometric convergence rate and supplement our theoretical analysis with experiments on synthetic data. Overall, we present a first thorough statistical and computational analysis of inference in the data-sharing model.
DT-PBO: an Interpretable Tree-based Surrogate Model for Preferential Bayesian Optimization
Data Science Center of Excellence, Faculty of Military Sciences, Breda, NL(数据科学卓越中心,军事科学学院,荷兰布雷达);NLR - Royal Netherlands Aerospace Centre, Amsterdam, NL(NLR-荷兰航空航天研究中心,荷兰阿姆斯特丹);Departement Intelligent Systems, Tilburg University, Tilburg, NL(智能系统部门,蒂尔堡大学,荷兰蒂尔堡)
Fractional ridge regression: a fast, interpretable reparameterization of ridge regression
The University of Washington eScience Institute(华盛顿大学电子科学研究所);University of Washington(华盛顿大学);Center for Magnetic Resonance Research(磁共振研究中心);University of Minnesota(明尼苏达大学)