Multi-task learning (MTL) aims at improving the generalization performance of several related tasks by leveraging useful information contained in them. However, in industrial scenarios, interpretability is always demanded, and the data of different tasks may be in heterogeneous domains, making the existing methods unsuitable or unsatisfactory. In this paper, following the philosophy of boosted tree, we proposed a two-stage method. In stage one, a common model is built to learn the commonalities using the common features of all instances. Different from the training of conventional boosted tree model, we proposed a regularization strategy and an early-stopping mechanism to optimize the multi-task learning process. In stage two, started by fitting the residual error of the common model, a specific model is constructed with the task-specific instances to further boost the performance. Experiments on both benchmark and real-world datasets validate the effectiveness of the proposed method. What's more, interpretability can be naturally obtained from the tree based method, satisfying the industrial needs.
We propose to use boosted regression trees as a way to compute human-interpretable solutions to reinforcement learning problems. Boosting combines several regression trees to improve their accuracy without significantly reducing their inherent interpretability. Prior work has focused independently on reinforcement learning and on interpretable machine learning, but there has been little progress in interpretable reinforcement learning. Our experimental results show that boosted regression trees compute solutions that are both interpretable and match the quality of leading reinforcement learnin…
Ensembles of decision trees perform well on many problems, but are not interpretable. In contrast to existing approaches in interpretability that focus on explaining relationships between features and predictions, we propose an alternative approach to interpret tree ensemble classifiers by surfacing representative points for each class -- prototypes. We introduce a new distance for Gradient Boosted Tree models, and propose new, adaptive prototype selection methods with theoretical guarantees, with the flexibility to choose a different number of prototypes in each class. We demonstrate our meth…
ISAM-MTL: Cross-subject multi-task learning model with identifiable spikes and associative memory networks
School of Future Technology, South China University of Technology（华南理工大学未来技术学院）；School of Artificial Intelligence and Automation, Huazhong University of Science and Technology（华中科技大学人工智能与自动化学院）
University of Sheffield（谢菲尔德大学）；The Alan Turing Institute（艾伦·图灵研究所）；University of Cambridge（剑桥大学）
There have been recent efforts to move to population-based structural health monitoring (PBSHM) systems. One area of PBSHM which has been recognised for potential development is the use of multi-task learning (MTL); algorithms which differ from traditional independent learning algorithms. Presented here is the use of the MTL, ''Joint Feature Selection with LASSO'', to provide automatic feature selection for a structural dataset. The classification task is to differentiate between the port and starboard side of a tailplane, for samples from two aircraft of the same model. The independent learne…
Gradient boosted decision trees are a popular machine learning technique, in part because of their ability to give good accuracy with small models. We describe two extensions to the standard tree boosting algorithm designed to increase this advantage. The first improvement extends the boosting formalism from scalar-valued trees to vector-valued trees. This allows individual trees to be used as multiclass classifiers, rather than requiring one tree per class, and drastically reduces the model size required for multiclass problems. We also show that some other popular vector-valued gradient boos…
Many autonomous systems, such as robots and self-driving cars, involve real-time decision making in complex environments, and require prediction of future outcomes from limited data. Moreover, their decisions are increasingly required to be interpretable to humans for safe and trustworthy co-existence. This paper is a first step towards interpretable learning-based robot control. We introduce a novel learning problem, called incremental formula and predictor learning, to generate binary classifiers with temporal logic structure from time-series data. The classifiers are represented as pairs of…
MTL-MAD: Multi-Task Learners are Effective Medical Anomaly Detectors
Rayscape；University of Bucharest（布加勒斯特大学）；Polytechnic University of Timişoara（蒂米什瓦拉工业大学）；Colţea Clinical Hospital（科尔切亚医院）；Carol Davila University of Medicine and Pharmacy（卡罗尔·迪瓦ila医学院和药理学大学）
Indian Institute of Technology Madras（马德拉斯印度理工学院）；The University of Texas at Dallas（德克萨斯大学达拉斯分校）
Contextual bandits algorithms have become essential in real-world user interaction problems in recent years. However, these algorithms rely on context as attribute value representation, which makes them unfeasible for real-world domains like social networks are inherently relational. We propose Relational Boosted Bandits(RB2), acontextual bandits algorithm for relational domains based on (relational) boosted trees. RB2 enables us to learn interpretable and explainable models due to the more descriptive nature of the relational representation. We empirically demonstrate the effectiveness and in…
Tree ensembles such as random forests and boosted trees are accurate but difficult to understand, debug and deploy. In this work, we provide the inTrees (interpretable trees) framework that extracts, measures, prunes and selects rules from a tree ensemble, and calculates frequent variable interactions. An rule-based learner, referred to as the simplified tree ensemble learner (STEL), can also be formed and used for future prediction. The inTrees framework can applied to both classification and regression problems, and is applicable to many types of tree ensembles, e.g., random forests, regular…
Multi-Task Learning Based on Support Vector Machines and Twin Support Vector Machines: A Comprehensive Survey
Department of Computer Science, Faculty of Mathematical Sciences, Alzahra University（计算机科学系，数学科学学院，阿扎赫拉大学）；Department of Informatics, Faculty of Science, Jan Evangelista Purkyne University（信息系，科学学院，扬·埃万杰利斯塔·普日科夫斯基大学）；Department of Computer Engineering, Science and Research Branch, IAU（计算机工程系，科学与研究分支，伊阿大学）；Department of Industrial and Systems Engineering, University of Florida（工业与系统工程系，佛罗里达大学）
Interpretable Boosted Decision Tree Analysis for the Majorana Demonstrator
Pacific Northwest National Laboratory（太平洋西北国家实验室）；University of South Carolina（南卡罗来纳大学）；Oak Ridge National Laboratory（橡树岭国家实验室）；National Research Center “Kurchatov Institute”（库尔恰托夫研究所）；Institute for Theoretical and Experimental Physics（理论与实验物理研究所）；University of South Dakota（南达科他大学）；University of North Carolina（北卡罗来纳大学）；Triangle Universities Nuclear Laboratory（三角大学核实验室）；North Carolina State University（北卡罗来纳州立大学）；Duke University（杜克大学）；SLAC National Accelerator Laboratory（SLAC国家加速器实验室）；University of Washington（华盛顿大学）
Know-Center GmbH（Know-Center有限公司）；Materials Center Leoben Forschung GmbH（莱奥本材料研究中心有限公司）
In this paper, we frame homogeneous-feature multi-task learning (MTL) as a hierarchical representation learning problem, with one task-agnostic and multiple task-specific latent representations. Drawing inspiration from the information bottleneck principle and assuming an additive independent noise model between the task-agnostic and task-specific latent representations, we limit the information contained in each task-specific representation. It is shown that our resulting representations yield competitive performance for several MTL benchmarks. Furthermore, for certain setups, we show that th…
Scaling regression to large datasets is a common problem in many application areas. We propose a two step approach to scaling regression to large datasets. Using a regression tree (CART) to segment the large dataset constitutes the first step of this approach. The second step of this approach is to develop a suitable regression model for each segment. Since segment sizes are not very large, we have the ability to apply sophisticated regression techniques if required. A nice feature of this two step approach is that it can yield models that have good explanatory power as well as good predictive…
Weakly Labelled learning has garnered lot of attention in recent years due to its potential to scale Sound Event Detection (SED) and is formulated as Multiple Instance Learning (MIL) problem. This paper proposes a Multi-Task Learning (MTL) framework for learning from Weakly Labelled Audio data which encompasses the traditional MIL setup. To show the utility of proposed framework, we use the input TimeFrequency representation (T-F) reconstruction as the auxiliary task. We show that the chosen auxiliary task de-noises internal T-F representation and improves SED performance under noisy recording…
Clinical QA 2.0: Multi-Task Learning for Answer Extraction and Categorization
University of Washington（华盛顿大学）；New York University（纽约大学）；Liverpool John Moores University（利物浦约翰·摩尔斯大学）；Birla Institute of Technology Pilani（比拉理工学院帕利尼分校）；Columbia University（哥伦比亚大学）
Inductive inference of gradient-boosted decision trees on graphs for insurance fraud detection
Data Office, Allianz Benelux（Allianz Benelux 数据办公室）；KU Leuven University of Antwerp-imec（安特卫普-imec 联合大学KU Leuven）；KU Leuven；University of Antwerp-imec（安特卫普-imec 大学）
Department of Artificial Intelligence, University of Malta（马耳他大学人工智能系）
With the ever-growing achievements in Artificial Intelligence (AI) and the recent boosted enthusiasm in Financial Technology (FinTech), applications such as credit scoring have gained substantial academic interest. Credit scoring helps financial experts make better decisions regarding whether or not to accept a loan application, such that loans with a high probability of default are not accepted. Apart from the noisy and highly imbalanced data challenges faced by such credit scoring models, recent regulations such as the `right to explanation' introduced by the General Data Protection Regulati…
IIT Madras（印度马德拉斯理工学院）；Robert Bosch Centre for Data Science and AI（罗伯特·博世数据科学与人工智能中心）
Models often need to be constrained to a certain size for them to be considered interpretable. For example, a decision tree of depth 5 is much easier to understand than one of depth 50. Limiting model size, however, often reduces accuracy. We suggest a practical technique that minimizes this trade-off between interpretability and classification accuracy. This enables an arbitrary learning algorithm to produce highly accurate small-sized models. Our technique identifies the training data distribution to learn from that leads to the highest accuracy for a model of a given size. We represent the …
