Apart from the high accuracy of machine learning models, what interests many researchers in real-life problems (e.g., fraud detection, credit scoring) is to find hidden patterns in data; particularly when dealing with their challenging imbalanced characteristics. Interpretability is also a key requirement that needs to accompany the used machine learning model. In this concern, often, intrinsically interpretable models are preferred to complex ones, which are in most cases black-box models. Also, linear models are used in some high-risk fields to handle tabular data, even if performance must be sacrificed. In this paper, we introduce Self-Reinforcement Attention (SRA), a novel attention mechanism that provides a relevance of features as a weight vector which is used to learn an intelligible representation. This weight is then used to reinforce or reduce some components of the raw input through element-wise vector multiplication. Our results on synthetic and real-world imbalanced data show that our proposed SRA block is effective in end-to-end combination with baseline models.
Department of Electronic Science, Xiamen University（厦门大学电子科学系）
We present a general architecture of deep differentiable forest and its sparse attention mechanism. The differentiable forest has the advantages of both trees and neural networks. Its structure is a simple binary tree, easy to use and understand. It has full differentiability and all variables are learnable parameters. We would train it by the gradient-based optimization method, which shows great power in the training of deep CNN. We find and analyze the attention mechanism in the differentiable forest. That is, each decision depends on only a few important features, and others are irrelevant.…
Understanding the semantic meaning of tabular data requires Entity Linking (EL), in order to associate each cell value to a real-world entity in a Knowledge Base (KB). In this work, we focus on end-to-end solutions for EL on tabular data that do not rely on fact lookup in the target KB. Tabular data contains heterogeneous and sparse context, including column headers, cell values and table captions. We experiment with various models to generate a vector representation for each cell value to be linked. Our results show that it is critical to apply an attention mechanism as well as an attention m…
Attention-Based Deep Learning for Early Parkinson's Disease Detection with Tabular Biomedical Data
African Institute for Mathematical Sciences, Kigali, Rwanda（非洲数学科学研究所，基加利，刚果（金））；Department of Data and Information Science, University of Ibadan, Oyo State, Nigeria（数据与信息科学系，伊巴丹大学，奥约州，尼日利亚）；Department of Computer Science and Information Systems, University of Limerick, Castletroy, V94 T9PX, Limerick, Ireland（计算机科学与信息系统系，利默里克大学，卡斯利特罗伊，爱尔兰）
Explainable Dual-Attention Tabular Transformer for Soil Electrical Resistivity Prediction: A Decision Support Framework for High-Voltage Substation Construction
King Mongkut’s University of technology Thonburi（泰国国王科技大学）
本研究提出一种用于预测土壤电阻率的双注意力 Transformer 架构，结合粒子群优化调参，在保持高预测精度的同时通过 SHAP 值提供可解释性，并开发了辅助高压变电站建设的 Web 决策支持框架。
The best neural architecture for a given machine learning problem depends on many factors: not only the complexity and structure of the dataset, but also on resource constraints including latency, compute, energy consumption, etc. Neural architecture search (NAS) for tabular datasets is an important but under-explored problem. Previous NAS algorithms designed for image search spaces incorporate resource constraints directly into the reinforcement learning (RL) rewards. However, for NAS on tabular datasets, this protocol often discovers suboptimal architectures. This paper develops TabNAS, a ne…
GeoViSTA: Geospatial Vision-Tabular Transformer for Multimodal Environment Representation
Department of Electrical and Computer Engineering, Rice University（理海大学电气与计算机工程系）；Center for Cardiovascular Computational and Precision Health, Department of Cardiology, DeBakey Heart and Vascular Center, Houston Methodist（休斯顿方法主义医疗中心心血管计算与精准健康中心、心内科部门、德贝基心脏和血管中心）
There is an increasing interest in the application of deep learning architectures to tabular data. One of the state-of-the-art solutions is TabTransformer which incorporates an attention mechanism to better track relationships between categorical features and then makes use of a standard MLP to output its final logits. In this paper we propose multiple modifications to the original TabTransformer performing better on binary classification tasks for three separate datasets with more than 1% AUROC gains. Inspired by gated MLP, linear projections are implemented in the MLP block and multiple acti…
Can Tabular Foundation Models Amortize Statistical Inference?
London School of Economics and Political Science（伦敦政治经济学院）；University of Science and Technology of China（中国科学技术大学）；Shanghai University of Finance and Economics（上海财经大学）；University of Milano-Bicocca（米兰比可卡大学）
University of Waterloo（滑铁卢大学）；University of Alberta（阿尔伯塔大学）
Some reinforcement learning methods suffer from high sample complexity causing them to not be practical in real-world situations. $Q$-function reuse, a transfer learning method, is one way to reduce the sample complexity of learning, potentially improving usefulness of existing algorithms. Prior work has shown the empirical effectiveness of $Q$-function reuse for various environments when applied to model-free algorithms. To the best of our knowledge, there has been no theoretical work showing the regret of $Q$-function reuse when applied to the tabular, model-free setting. We aim to bridge th…
Meticulously analysing the empirical strengths and weaknesses of reinforcement learning methods in hard (challenging) environments is essential to inspire innovations and assess progress in the field. In tabular reinforcement learning, there is no well-established standard selection of environments to conduct such analysis, which is partially due to the lack of a widespread understanding of the rich theory of hardness of environments. The goal of this paper is to unlock the practical usefulness of this theory through four main contributions. First, we present a systematic survey of the theory …
A Unified Algorithmic Framework for Hybrid Reinforcement Learning in Tabular MDPs with Shifted Transition Dynamics
Southern University of Science and Technology（南方科技大学）；Harbin Institute of Technology, Shenzhen（哈尔滨工业大学（深圳））；City University of Hong Kong（香港城市大学）；Fudan University（复旦大学）；Shanghai Academy of AI for Science（上海人工智能研究院）
Towards Data-Centric AI: A Comprehensive Survey of Traditional, Reinforcement, and Generative Approaches for Tabular Data Transformation
University of Kansas（堪萨斯大学）；Southwestern University of Finance and Economics（西南财经大学）；Arizona State University（亚利桑那州立大学）；Portland State University（波特兰州立大学）；Computer Network Information Center, Chinese Academy of Sciences（中国科学院计算机网络信息中心）；University of Macau（澳门大学）；The Hong Kong University of Science and Technology（香港科技大学）
Tsinghua University（清华大学）；University of Washington（华盛顿大学）
This paper gives the first polynomial-time algorithm for tabular Markov Decision Processes (MDP) that enjoys a regret bound \emph{independent on the planning horizon}. Specifically, we consider tabular MDP with $S$ states, $A$ actions, a planning horizon $H$, total reward bounded by $1$, and the agent plays for $K$ episodes. We design an algorithm that achieves an $O\left(\mathrm{poly}(S,A,\log K)\sqrt{K}\right)$ regret in contrast to existing bounds which either has an additional $\mathrm{polylog}(H)$ dependency~\citep{zhang2020reinforcement} or has an exponential dependency on $S$~\citep{li2…
Causal-Aware Tabular GANs with Reinforcement Learning
Applied Artificial Intelligence Initiative (A 2 I 2 ), Deakin University, Australia（应用人工智能倡议（A2I2），德肯大学，澳大利亚）；School of Computer Science and Information Technology, Adelaide University, Australia（计算机科学与信息科技学院，阿德莱德大学，澳大利亚）
Sample-Efficient Tabular Self-Play for Offline Robust Reinforcement Learning
Zhejiang University（浙江大学）；The Chinese University of Hong Kong（香港中文大学）；Edith Cowan University（埃迪斯·科温大学）；University of New South Wales（新南威尔士大学）；Huazhong University of Science and Technology（华中科技大学）
University of Maryland, College Park（马里兰大学帕克分校）；Capital One（第一资本金融公司）
Tabular data underpins numerous high-impact applications of machine learning from fraud detection to genomics and healthcare. Classical approaches to solving tabular problems, such as gradient boosting and random forests, are widely used by practitioners. However, recent deep learning methods have achieved a degree of performance competitive with popular techniques. We devise a hybrid deep learning approach to solving tabular data problems. Our method, SAINT, performs attention over both rows and columns, and it includes an enhanced embedding method. We also study a new contrastive self-superv…
