There have been recent efforts to move to population-based structural health monitoring (PBSHM) systems. One area of PBSHM which has been recognised for potential development is the use of multi-task learning (MTL); algorithms which differ from traditional independent learning algorithms. Presented here is the use of the MTL, ''Joint Feature Selection with LASSO'', to provide automatic feature selection for a structural dataset. The classification task is to differentiate between the port and starboard side of a tailplane, for samples from two aircraft of the same model. The independent learner produced perfect F1 scores but had poor engineering insight; whereas the MTL results were interpretable, highlighting structural differences as opposed to differences in experimental set-up.
Central University of Finance and Economics（中央财经大学）；University of York（约克大学）；University of Illinois at Chicago（伊利诺伊大学芝加哥分校）
Feature selection has been proven a powerful preprocessing step for high-dimensional data analysis. However, most state-of-the-art methods tend to overlook the structural correlation information between pairwise samples, which may encapsulate useful information for refining the performance of feature selection. Moreover, they usually consider candidate feature relevancy equivalent to selected feature relevancy, and some less relevant features may be misinterpreted as salient features. To overcome these issues, we propose a new feature selection method using structural correlation between pairw…
College of Software, Jilin University（吉林大学软件学院）；College of Computer Science and Technology, Jilin University（吉林大学计算机科学与技术学院）；Key Laboratory of Symbolic Computation and Knowledge Engineering of the Ministry of Education（教育部符号计算与知识工程重点实验室）；School of Computing and Artificial Intelligence, Southwestern University of Finance and Economics（西南财经大学计算机与人工智能学院）；Engineering Research Center of Intelligent Finance, Ministry of Education（教育部智能金融工程研究中心）；Shenzhen Key Laboratory of Visual Object Detection and Recognition, Harbin Institute of Technology（哈尔滨工业大学深圳视觉对象检测与识别重点实验室）；School of Artificial Intelligence and Computer Science, Nantong University（南通大学人工智能与计算机科学学院）；Faculty of Data Science, City University of Macau（澳门城市大学数据科学学院）
Aalto University（阿尔托大学）；QPR Software Plc（QPR软件公司）；HIIT Helsinki Institute for Information Technology（赫尔辛基信息技术研究所）
We consider the problem of classifying business process instances based on structural features derived from event logs. The main motivation is to provide machine learning based techniques with quick response times for interactive computer assisted root cause analysis. In particular, we create structural features from process mining such as activity and transition occurrence counts, and ordering of activities to be evaluated as potential features for classification. We show that adding such structural features increases the amount of information thus potentially increasing classification accura…
Central University of Finance and Economics（中央财经大学）；Xiamen University（厦门大学）；University of York（约克大学）
Feature selection can efficiently identify the most informative features with respect to the target feature used in training. However, state-of-the-art vector-based methods are unable to encapsulate the relationships between feature samples into the feature selection process, thus leading to significant information loss. To address this problem, we propose a new graph-based structurally interacting elastic net method for feature selection. Specifically, we commence by constructing feature graphs that can incorporate pairwise relationship between samples. With the feature graphs to hand, we pro…
We present a new latent-variable model employing a Gaussian mixture integrated with a feature selection procedure (the Bernoulli part of the model) which together form a "Latent Bernoulli-Gauss" distribution. The model is applied to MAP estimation, clustering, feature selection and collaborative filtering and fares favorably with the state-of-the-art latent-variable models.
Tsinghua University（清华大学）；University of Texas Austin（德克萨斯大学奥斯汀分校）
Feature selection is a process of choosing a subset of relevant features so that the quality of prediction models can be improved. An extensive body of work exists on information-theoretic feature selection, based on maximizing Mutual Information (MI) between subsets of features and class labels. The prior methods use a lower order approximation, by treating the joint entropy as a summation of several single variable entropies. This leads to locally optimal selections and misses multi-way feature combinations. We present a higher order MI based approximation technique called Higher Order Featu…
Bosch Center for Artificial Intelligence（博世人工智能中心）
Despite recent advances in automated machine learning, model selection is still a complex and computationally intensive process. For Gaussian processes (GPs), selecting the kernel is a crucial task, often done manually by the expert. Additionally, evaluating the model selection criteria for Gaussian processes typically scales cubically in the sample size, rendering kernel search particularly computationally expensive. We propose a novel, efficient search method through a general, structured kernel space. Previous methods solved this task via Bayesian optimization and relied on measuring the di…
Harvard University（哈佛大学）；University of Michigan（密歇根大学）
Motivated by the need for efficient and personalized learning in mobile health, we investigate the problem of online kernel selection for Gaussian Process regression in the multi-task setting. We propose a novel generative process on the kernel composition for this purpose. Our method demonstrates that trajectories of kernel evolutions can be transferred between users to improve learning and that the kernels themselves are meaningful for an mHealth prediction goal.
Combining Textual and Structural Information for Premise Selection in Lean
Faculty of Mathematics and Physics, University of Ljubljana（卢布尔雅纳大学数学与物理学院）；University of Ljubljana（卢布尔雅纳大学）；Department of Knowledge Technologies, Jožef Stefan Institute（知识技术系，乔泽夫·斯塔芬研究所）
Structural-Entropy-Based Sample Selection for Efficient and Effective Learning
Tsinghua University（清华大学）；China Telecom Wanwei Information Technology Co., Ltd（中国电信万维信息技术有限公司）；Microsoft Research（微软研究院）；Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州））
University of Sheffield（谢菲尔德大学）；University of Melbourne（墨尔本大学）；Uppsala University（乌普萨拉大学）
Structural kernels are a flexible learning paradigm that has been widely used in Natural Language Processing. However, the problem of model selection in kernel-based methods is usually overlooked. Previous approaches mostly rely on setting default values for kernel hyperparameters or using grid search, which is slow and coarse-grained. In contrast, Bayesian methods allow efficient model selection by maximizing the evidence on the training data through gradient-based methods. In this paper we show how to perform this in the context of structural kernels by using Gaussian Processes. Experimental…
In recent years there has been a flurry of works on learning Bayesian networks from data. One of the hard problems in this area is how to effectively learn the structure of a belief network from incomplete data- that is, in the presence of missing values or hidden variables. In a recent paper, I introduced an algorithm called Structural EM that combines the standard Expectation Maximization (EM) algorithm, which optimizes parameters, with structure search for model selection. That algorithm learns networks based on penalized likelihood scores, which include the BIC/MDL score and various approx…
Informatics Institute（信息学研究所）；University of Amsterdam（阿姆斯特丹大学）
We prove the main rules of causal calculus (also called do-calculus) for i/o structural causal models (ioSCMs), a generalization of a recently proposed general class of non-/linear structural causal models that allow for cycles, latent confounders and arbitrary probability distributions. We also generalize adjustment criteria and formulas from the acyclic setting to the general one (i.e. ioSCMs). Such criteria then allow to estimate (conditional) causal effects from observational data that was (partially) gathered under selection bias and cycles. This generalizes the backdoor criterion, the se…
Informatics Institute（信息学研究所）；University of Amsterdam（阿姆斯特丹大学）
Despite their popularity, many questions about the algebraic constraints imposed by linear structural equation models remain open problems. For causal discovery, two of these problems are especially important: the enumeration of the constraints imposed by a model, and deciding whether two graphs define the same statistical model. We show how the half-trek criterion can be used to make progress in both of these problems. We apply our theoretical results to a small-scale model selection problem, and find that taking the additional algebraic constraints into account may lead to significant improv…
University of Illinois, Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）
This paper addresses the problem of neighborhood selection for Gaussian graphical models. We present two heuristic algorithms: a forward-backward greedy algorithm for general Gaussian graphical models based on mutual information test, and a threshold-based algorithm for walk summable Gaussian graphical models. Both algorithms are shown to be structurally consistent, and efficient. Numerical results show that both algorithms work very well.
Structural optimization is a popular method for designing objects such as bridge trusses, airplane wings, and optical devices. Unfortunately, the quality of solutions depends heavily on how the problem is parameterized. In this paper, we propose using the implicit bias over functions induced by neural networks to improve the parameterization of structural optimization. Rather than directly optimizing densities on a grid, we instead optimize the parameters of a neural network which outputs those densities. This reparameterization leads to different and often better solutions. On a selection of …
Causal processes in nature may contain cycles, and real datasets may violate causal sufficiency as well as contain selection bias. No constraint-based causal discovery algorithm can currently handle cycles, latent variables and selection bias (CLS) simultaneously. I therefore introduce an algorithm called Cyclic Causal Inference (CCI) that makes sound inferences with a conditional independence oracle under CLS, provided that we can represent the cyclic causal process as a non-recursive linear structural equation model with independent errors. Empirical results show that CCI outperforms CCD in …
Geneva School of Business Administration, HES-SO（日内瓦工商管理学院，HES-SO）；University of Geneva（日内瓦大学）
We propose a new method for input variable selection in nonlinear regression. The method is embedded into a kernel regression machine that can model general nonlinear functions, not being a priori limited to additive models. This is the first kernel-based variable selection method applicable to large datasets. It sidesteps the typical poor scaling properties of kernel methods by mapping the inputs into a relatively low-dimensional space of random features. The algorithm discovers the variables relevant for the regression task together with learning the prediction model through learning the app…
