Decision trees are widely used for classification and regression tasks in a variety of application fields due to their interpretability and good accuracy. During the past decade, growing attention has been devoted to globally optimized decision trees with deterministic or soft splitting rules at branch nodes, which are trained by optimizing the error function over all the tree parameters. In this work, we propose a new variant of soft multivariate regression trees (SRTs) where, for every input vector, the prediction is defined as the linear regression associated to a single leaf node, namely, the leaf node obtained by routing the input vector from the root along the branches with higher probability. SRTs exhibit the conditional computational property, i.e., each prediction depends on a small number of nodes (parameters), and our nonlinear optimization formulation for training them is amenable to decomposition. After showing a universal approximation result for SRTs, we present a decomposition training algorithm including a clustering-based initialization procedure and a heuristic for reassigning the input vectors along the tree. Under mild assumptions, we establish asymptotic convergence guarantees. Experiments on 15 wellknown datasets indicate that our SRTs and decomposition algorithm yield higher accuracy and robustness compared with traditional soft regression trees trained using the nonlinear optimization formulation of Blanquero et al., and a significant reduction in training times as well as a slightly better average accuracy compared with the mixed-integer optimization approach of Bertsimas and Dunn. We also report a comparison with the Random Forest ensemble method.
High-Dimensional Gaussian Process Regression with Soft Kernel Interpolation
Department of Computing and Mathematical Sciences, California Institute of Technology(计算与数学科学部,加州理工学院);Department of Computer Science, San Francisco State University(计算机科学部,旧金山州立大学);Base26, CA, USA
Classifying Phonotrauma Severity from Vocal Fold Images with Soft Ordinal Regression
Massachusetts Institute of Technology(麻省理工学院);School of Communication Sciences and Disorders University of Central Florida(中央佛罗里达大学通信科学与障碍学系);Harvard Medical School(哈佛医学院);Massachusetts General Hospital(麻省总医院);Center for Laryngeal Surgery and Voice Rehabilitation(嗓音外科与嗓音康复中心);Department of Surgery(外科部门);Institute of Health Professions(健康专业研究所);Athinoula Martinos Center(阿提尼奥拉马丁诺斯中心)
Adaptive Sparse Softmax: An Effective and Efficient Softmax Variant
School of Computer Science and Technology, Soochow University(苏州大学计算机科学与技术学院);School of Computer Science and Technology, Harbin Institute of Technology (Shenzhen)(哈尔滨工业大学(深圳)计算机科学与技术学院);School of Computer Science and Technology, Institution of Artificial Intelligence, Soochow University(苏州大学计算机科学与技术学院、人工智能研究所);Peking University(北京大学);Hong Kong Polytechnic University(香港理工大学)
Robust Gaussian Process Regression Based on Iterative Trimming
Shanghai Jiao Tong University(上海交通大学);Shanghai Astronomical Observatory, Chinese Academy of Sciences(中国科学院上海天文台);University of the Chinese Academy of Sciences(中国科学院大学);Key Lab for Astrophysics(天体物理重点实验室)
The Importance of Landscape Features for Performance Prediction of Modular CMA-ES Variants
Jožef Stefan Institute(约泽夫·斯蒂芬研究所);Jožef Stefan International Postgraduate School(约泽夫·斯蒂芬国际研究生院);Leiden Institute for Advanced Computer Science(莱顿高级计算机科学研究所);Sorbonne Université(索邦大学);CNRS(法国国家科学研究中心)
Optimal Algorithms for Ridge and Lasso Regression with Partially Observed Attributes
Technion — Israel Institute of Technology(以色列理工学院)
我们考虑线性回归的最常见变体,包括Ridge,Lasso和Support vector回归,在这种设置中,学习者在训练时只允许观察每个示例的固定数量的属性。我们提出了这些问题的简单而有效的算法:对于Lasso和Ridge回归,它们需要与全信息算法相同的属性总数(直到常数),以达到一定的精度。对于支持向量回归,与现有技术相比,我们需要指数级更少的属性。通过这一点,我们解决了Cesa-Bianchi et al.(2010)最近提出的一个开放问题。实验表明,与现有技术相比,优越的性能证明了理论界限的合理性。
Technion — Israel Institute of Technology(以色列理工学院)
我们考虑线性回归的最常见变体,包括Ridge,Lasso和Support vector回归,在这种设置中,学习者在训练时只允许观察每个示例的固定数量的属性。我们提出了这些问题的简单而有效的算法:对于Lasso和Ridge回归,它们需要与全信息算法相同的属性总数(直到常数),以达到一定的精度。对于支持向量回归,与现有技术相比,我们需要指数级更少的属性。通过这一点,我们解决了Cesa-Bianchi et al.(2010)最近提出的一个开放问题。实验表明,与现有技术相比,优越的性能证明了理论界限的合理性。
Mutual Information Learned Regressor: an Information-theoretic Viewpoint of Training Regression Systems
University of Iowa(爱荷华大学);National University of Singapore(新加坡国立大学);University of California at Irvine(加州大学欧文分校);Stanford University(斯坦福大学);University of Florida(佛罗里达大学);Grinnell College(格林内尔学院)