An extension of the regularized least-squares in which the estimation parameters are stretchable is introduced and studied in this paper. The solution of this ridge regression with stretchable parameters is given in primal and dual spaces and in closed-form. Essentially, the proposed solution stretches the covariance computation by a power term, thereby compressing or amplifying the estimation parameters. To maintain the computation of power root terms within the real space, an input transformation is proposed. The results of an empirical evaluation in both synthetic and real-world data illustrate that the proposed method is effective for compressive learning with high-dimensional data.
A Deterministic Streaming Sketch for Ridge Regression
University of Utah(犹他大学)
我们提供了一个确定性的空间有效算法估计岭回归。对于具有$d$特征和足够大的正则化参数的$n$数据点,我们仅使用$O(d/\varepsilon)$空间提供了$\varepsilon$ L $_2$误差内的解决方案。这是第一个$o(d^2)$空间确定性流算法,保证解决这个经典问题的错误和风险界。该算法通过频繁方向的变体来勾画协方差矩阵,这意味着它可以在仅插入流和各种分布式数据设置中操作。在合成和真实世界的数据集上的随机草图算法相比,我们的算法具有较少的经验误差,使用较少的空间和相似的时间。
SLEIPNIR: Deterministic and Provably Accurate Feature Expansion for Gaussian Process Regression with Derivatives
ETH Zürich(苏黎世联邦理工学院);Max Planck ETH Center for Learning Systems(马克斯·普朗克-苏黎世联邦理工学院学习系统中心);Max Planck Institute for Intelligent Systems, Tübingen(图宾根马克斯·普朗克智能系统研究所)
Two-Point Deterministic Equivalence for Stochastic Gradient Dynamics in Linear Models
John A. Paulson School of Engineering and Applied Sciences, Harvard University(哈佛大学约翰·A·保罗森工程与应用科学学校);Center of Mathematical Sciences and Applications, Harvard University(哈佛大学数学科学中心);Society of Fellows, Harvard University(哈佛大学 fellows 会);Center for Brain Science, Harvard University(哈佛大学脑科学中心);Department of Mathematics and Statistics, McGill University(麦吉尔大学数学与统计学系);Kempner Institute for the Study of Natural and Artificial Intelligence, Harvard University(哈佛大学自然与人工智能研究 institute)
Symbolic Regression with Fast Function Extraction and Nonlinear Least Squares Optimization
Josef Ressel Center for Symbolic Regression(约瑟夫·雷塞尔符号回归中心);University of Applied Sciences Upper Austria(上奥地利应用科学大学);Johannes Kepler University(约翰内斯·开普勒大学)
COIN-GP: Cooperative Online Learning in Networked Distributed Systems with Partial Measurements via Gaussian Process Regression
Technical University of Munich(慕尼黑工业大学);The Hong Kong University of Science and Technology (Guangzhou)(香港科技大学(广州));Tongji University(同济大学);Shanghai Yangzhi Rehabilitation Hospital(上海市养志康复医院);University of Science and Technology of China(中国科学技术大学);The Hong Kong Polytechnic University(香港理工大学)