Combinatorial optimization lies at the core of many real-world problems. Especially since the rise of graph neural networks (GNNs), the deep learning community has been developing solvers that derive solutions to NP-hard problems by learning the problem-specific solution structure. However, reproducing the results of these publications proves to be difficult. We make three contributions. First, we present an open-source benchmark suite for the NP-hard Maximum Independent Set problem, in both its weighted and unweighted variants. The suite offers a unified interface to various state-of-the-art traditional and machine learning-based solvers. Second, using our benchmark suite, we conduct an in-depth analysis of the popular guided tree search algorithm by Li et al. [NeurIPS 2018], testing various configurations on small and large synthetic and real-world graphs. By re-implementing their algorithm with a focus on code quality and extensibility, we show that the graph convolution network used in the tree search does not learn a meaningful representation of the solution structure, and can in fact be replaced by random values. Instead, the tree search relies on algorithmic techniques like graph kernelization to find good solutions. Thus, the results from the original publication are not reproducible. Third, we extend the analysis to compare the tree search implementations to other solvers, showing that the classical algorithmic solvers often are faster, while providing solutions of similar quality. Additionally, we analyze a recent solver based on reinforcement learning and observe that for this solver, the GNN is responsible for the competitive solution quality.
From a safety perspective, a machine learning method embedded in real-world applications is required to distinguish irregular situations. For this reason, there has been a growing interest in the anomaly detection (AD) task. Since we cannot observe abnormal samples for most of the cases, recent AD methods attempt to formulate it as a task of classifying whether the sample is normal or not. However, they potentially fail when the given normal samples are inherited from diverse semantic labels. To tackle this problem, we introduce a latent class-condition-based AD scenario. In addition, we propo…
University of Toronto（多伦多大学）；Vector Institute（向量研究所）；Harvard University (SEAS)（哈佛大学工程与应用科学学院）；Lunenfeld Tanenbaum Research Institute（卢嫩菲尔德-塔南鲍姆研究所）；The Hospital for Sick Children（病童医院）
Explanations of time series models are useful for high stakes applications like healthcare but have received little attention in machine learning literature. We propose FIT, a framework that evaluates the importance of observations for a multivariate time-series black-box model by quantifying the shift in the predictive distribution over time. FIT defines the importance of an observation based on its contribution to the distributional shift under a KL-divergence that contrasts the predictive distribution against a counterfactual where the rest of the features are unobserved. We also demonstrat…
Cheriton School of Computer Science（滑铁卢大学计算机科学学院）；University of Waterloo（滑铁卢大学）；Waterloo, Canada（滑铁卢，加拿大（注：经检查，该条目为地址信息，根据规则应忽略，原输出错误，正确应为仅保留前两个机构））
Unsupervised learning is widely recognized as one of the most important challenges facing machine learning nowa- days. However, in spite of hundreds of papers on the topic being published every year, current theoretical understanding and practical implementations of such tasks, in particular of clustering, is very rudimentary. This note focuses on clustering. I claim that the most signif- icant challenge for clustering is model selection. In contrast with other common computational tasks, for clustering, dif- ferent algorithms often yield drastically different outcomes. Therefore, the choice o…
What's Wrong with Your Synthetic Tabular Data? Using Explainable AI to Evaluate Generative Models
Leibniz Institute for Prevention Research & Epidemiology – BIPS（莱比锡预防研究与流行病学研究所 – BIPS）；Faculty of Mathematics and Computer Science, University of Bremen（数学与计算机科学学院，不莱梅大学）；Norwegian Computing Center（挪威计算中心）
Technical University of Darmstadt（达姆施塔特工业大学）；MedCom GmbH（MedCom股份有限公司）
Continual learning protocols are attracting increasing attention from the medical imaging community. In continual environments, datasets acquired under different conditions arrive sequentially; and each is only available for a limited period of time. Given the inherent privacy risks associated with medical data, this setup reflects the reality of deployment for deep learning diagnostic radiology systems. Many techniques exist to learn continuously for image classification, and several have been adapted to semantic segmentation. Yet most struggle to accumulate knowledge in a meaningful manner. …
Within the last few years, there has been a move towards using statistical models in conjunction with neural networks with the end goal of being able to better answer the question, "what do our models know?". From this trend, classical metrics such as Prediction Interval Coverage Probability (PICP) and new metrics such as calibration error have entered the general repertoire of model evaluation in order to gain better insight into how the uncertainty of our model compares to reality. One important component of uncertainty modeling is model uncertainty (epistemic uncertainty), a measurement of …
What is Wrong with Perplexity for Long-context Language Modeling?
State Key Lab of General Artificial Intelligence, School of Intelligence Science and Technology, Peking University（一般人工智能国家重点实验室，智能科学与技术学院，北京大学）；MIT CSAIL（麻省理工学院计算机科学与人工智能实验室）；Alibaba Group（阿里巴巴集团）；TUM CIT, MCML, MDSI（技术大学（TUM） CIT、MCML、MDSI）；MIT EECS, CSAIL（麻省理工学院电子工程与计算机科学系，计算机科学与人工智能实验室）；Institute for Artificial Intelligence, Peking University（人工智能研究院，北京大学）
Institute of Biomedical Sciences, Academia Sinica（中央研究院生物医学科学研究所）
The paper discusses the limitations of deep learning models in identifying and utilizing features that remain invariant under a bijective transformation on the data entries, which we refer to as combinatorial patterns. We argue that the identification of such patterns may be important for certain applications and suggest providing neural networks with information that fully describes the combinatorial patterns of input entries and allows the network to determine what is relevant for prediction. To demonstrate the feasibility of this approach, we present a combinatorial convolutional neural net…
Stable but Wrong: When Learning Stabilizes Away from the Truth
China Mobile Research Institute（中国移动研究院）；China Mobile GBA (Greater Bay Area) Innovation Institute（中国移动广州湾创新院）；Institute of Microelectronics, Chinese Academy of Sciences（中国科学院微电子研究所）
In this review, we discuss approaches for learning causal structure from data, also called causal discovery. In particular, we focus on approaches for learning directed acyclic graphs (DAGs) and various generalizations which allow for some variables to be unobserved in the available data. We devote special attention to two fundamental combinatorial aspects of causal structure learning. First, we discuss the structure of the search space over causal graphs. Second, we discuss the structure of equivalence classes over causal graphs, i.e., sets of graphs which represent what can be learned from o…
Importance-weighted risk minimization is a key ingredient in many machine learning algorithms for causal inference, domain adaptation, class imbalance, and off-policy reinforcement learning. While the effect of importance weighting is well-characterized for low-capacity misspecified models, little is known about how it impacts over-parameterized, deep neural networks. This work is inspired by recent theoretical results showing that on (linearly) separable data, deep linear networks optimized by SGD learn weight-agnostic solutions, prompting us to ask, for realistic deep networks, for which man…
Université de Montréal（蒙特利尔大学）；Mila, Quebec Artificial Intelligence Institute（米拉-魁北克人工智能研究所）；École Polytechnique de Montréal（蒙特利尔综合理工学院）
This paper surveys the recent attempts, both from the machine learning and operations research communities, at leveraging machine learning to solve combinatorial optimization problems. Given the hard nature of these problems, state-of-the-art algorithms rely on handcrafted heuristics for making decisions that are otherwise too expensive to compute or mathematically not well defined. Thus, machine learning looks like a natural candidate to make such decisions in a more principled and optimized way. We advocate for pushing further the integration of machine learning and combinatorial optimizatio…
