While synthetic tabular data generation using Deep Generative Models (DGMs) offers a compelling solution to data scarcity and privacy concerns, their effectiveness relies on the availability of substantial training data, often lacking in real-world scenarios. To overcome this limitation, we propose a novel methodology that explicitly integrates artificial inductive biases into the generative process to improve data quality in low-data regimes. Our framework leverages transfer learning and meta-learning techniques to construct and inject informative inductive biases into DGMs. We evaluate four approaches (pre-training, model averaging, Model-Agnostic Meta-Learning (MAML), and Domain Randomized Search (DRS)) and analyze their impact on the quality of the generated text. Experimental results show that incorporating inductive bias substantially improves performance, with transfer learning methods outperforming meta-learning, achieving up to 60\% gains in Jensen-Shannon divergence. The methodology is model-agnostic and especially relevant in domains such as healthcare and finance, where high-quality synthetic data are essential, and data availability is often limited.
Knowledge-Data Dually Driven Paradigm for Accurate Landslide Susceptibility Prediction under Data-Scarce Conditions Using Geomorphic Priors and Tabular Foundation Model
School of Engineering and Technology, China University of Geosciences (Beijing)（工程与技术学院，中国地质大学（北京））；School of Geological Engineering and Geomatics, Chang'an University（地质工程与测绘学院，长安大学）
A challenging open question in deep learning is how to handle tabular data. Unlike domains such as image and natural language processing, where deep architectures prevail, there is still no widely accepted neural architecture that dominates tabular data. As a step toward bridging this gap, we present DNF-Net a novel generic architecture whose inductive bias elicits models whose structure corresponds to logical Boolean formulas in disjunctive normal form (DNF) over affine soft-threshold decision terms. In addition, DNF-Net promotes localized decisions that are taken over small subsets of the fe…
University of Maryland, College Park（马里兰大学帕克分校）；Capital One（第一资本金融公司）
Tabular data underpins numerous high-impact applications of machine learning from fraud detection to genomics and healthcare. Classical approaches to solving tabular problems, such as gradient boosting and random forests, are widely used by practitioners. However, recent deep learning methods have achieved a degree of performance competitive with popular techniques. We devise a hybrid deep learning approach to solving tabular data problems. Our method, SAINT, performs attention over both rows and columns, and it includes an enhanced embedding method. We also study a new contrastive self-superv…
TabGift: A Context-Efficient Foundation Model for Tabular Data Generation
School of Artificial Intelligence, Nanjing University（南京大学人工智能学院）；National Key Laboratory for Novel Software Technology, Nanjing University（南京大学计算机软件新技术国家重点实验室）
Topological Inductive Bias fosters Multiple Instance Learning in Data-Scarce Scenarios
Helmholtz Munich Technical University of Munich（海德堡-慕尼黑应用技术大学）；Ludwig-Maximilian-University Hospital University of Munich（路德维希-马克西米利安大学医院慕尼黑大学）；DKTK, German Cancer Consortium（德国癌症联合会）；Munich Center for Machine Learning（慕尼黑机器学习中心）；Helmholtz Munich University of Fribourg（海德堡-慕尼黑应用技术大学弗里堡大学）
XGenBoost: Synthesizing Small and Large Tabular Datasets with XGBoost
Public Health \& Primary Care, Leiden University Medical Center, Leiden, The Netherlands；Research；Development, Statistics Netherlands (CBS), The Hague, The Netherlands；Leiden Institute of Advanced Computer Science, Leiden University, Leiden, The Netherlands
Current deep learning based text classification methods are limited by their ability to achieve fast learning and generalization when the data is scarce. We address this problem by integrating a meta-learning procedure that uses the knowledge learned across many tasks as an inductive bias towards better natural language understanding. Based on the Model-Agnostic Meta-Learning framework (MAML), we introduce the Attentive Task-Agnostic Meta-Learning (ATAML) algorithm for text classification. The essential difference between MAML and ATAML is in the separation of task-agnostic representation lear…
Limited Reference, Reliable Generation: A Two-Component Framework for Tabular Data Generation in Low-Data Regimes
School of Computer Science, Fudan University（复旦大学计算机科学学院）；Institute of Financial Technology, Fudan University（复旦大学金融技术研究院）；Northwestern Polytechnical University（西北工业大学）
Can I trust my fake data -- A comprehensive quality assessment framework for synthetic tabular data in healthcare
University of Oslo（奥斯陆大学）；DNV AS（挪威船级社）；Cancer Registry of Norway（挪威癌症登记处）；Oslo University Hospital（奥斯陆大学医院）；UiT The Arctic University of Norway（挪威北极大学）
When Does Small Data Work? Accuracy and Efficiency Trade-offs Between Tabular Foundation Models and Conventional Methods for Crowd-State Classification at Hajj and Umrah
KnowsTFM: Knowledge-Informed Fine-Tuning of Small Tabular Foundation Models
Jožef Stefan Institute and Postgraduate School（乔塞夫·斯蒂芬研究所和研究生学院）；Department of Computer Science and Technology, University of Cambridge, UK（剑桥大学计算机科学与技术系）；Télécom Paris, Institut Polytechnique de Paris, FR（巴黎电信学院，巴黎理工学院）
ZAYAN: Disentangled Contrastive Transformer for Tabular Remote Sensing Data
Lane Dept. of Computer Science and Electrical Engineering（兰恩计算机科学与电气工程系）；Dept. of Geology & Geography（地质与地理系）；West Virginia University（西弗吉尼亚大学）；Department of Computer Science and Engineering（计算机科学与工程系）；Green University of Bangladesh（孟加拉国绿色大学）；Stamford University Bangladesh（班加罗尔斯坦福大学）
Pretrained deep models outperform GBDTs in Learning-To-Rank under label scarcity
Department of Electrical and Computer Engineering（电气与计算机工程系）；Carnegie Mellon University（卡内基梅隆大学）；Amazon（亚马逊）；Department of Computer Science（计算机科学系）；University of Texas at Austin（德克萨斯大学奥斯汀分校）
Google Research（谷歌研究院）；University of Washington（华盛顿大学）
We propose Distribution Embedding Networks (DEN) for classification with small data. In the same spirit of meta-learning, DEN learns from a diverse set of training tasks with the goal to generalize to unseen target tasks. Unlike existing approaches which require the inputs of training and target tasks to have the same dimension with possibly similar distributions, DEN allows training and target tasks to live in heterogeneous input spaces. This is especially useful for tabular-data tasks where labeled data from related tasks are scarce. DEN uses a three-block architecture: a covariate transform…
While deep learning has enabled tremendous progress on text and image datasets, its superiority on tabular data is not clear. We contribute extensive benchmarks of standard and novel deep learning methods as well as tree-based models such as XGBoost and Random Forests, across a large number of datasets and hyperparameter combinations. We define a standard set of 45 datasets from varied domains with clear characteristics of tabular data and a benchmarking methodology accounting for both fitting models and finding good hyperparameters. Results show that tree-based models remain state-of-the-art …
TabularQGAN: A quantum generative model for tabular data synthesis
SAP SE（SAP公司）；BASF Digital Solutions（巴斯夫数字解决方案）；QUTAC Quantum Technology and Application Consortium（QUTAC量子技术与应用联盟）；Technical University of Munich CIT（慕尼黑技术大学CIT）；Ludwig-Maximilians-Universität München（慕尼黑路德维希-马克西米利安大学）；Merck KGaA（默克公司）
QTabGAN: A Hybrid Quantum-Classical GAN for Tabular Data Synthesis
Department of Computer Science and Engineering, Indian Institute of Technology (BHU), Varanasi, India（计算机科学与工程系，印度理工学院（BHU），瓦拉纳西，印度）；Department of Applied Mathematics and Theoretical Physics, University of Cambridge, Cambridge, United Kingdom（应用数学与理论物理系，剑桥大学，剑桥，英国）
École Polytechnique Fédérale de Lausanne（洛桑联邦理工学院）
Synthetic data can be used in various applications, such as correcting bias datasets or replacing scarce original data for simulation purposes. Generative Adversarial Networks (GANs) are considered state-of-the-art for developing generative models. However, these deep learning models are data-driven, and it is, thus, difficult to control the generation process. It can, therefore, lead to the following issues: lack of representativity in the generated data, the introduction of bias, and the possibility of overfitting the sample's noise. This article presents the Directed Acyclic Tabular GAN (DA…
Rethinking Distribution Shifts: Empirical Analysis and Modeling for Tabular Data
Department of Industrial Engineering and Operations Research（工业工程与运筹学系）；Department of Computer Science and Technology（计算机科学与技术系）；Decision, Risk, and Operations Division（决策、风险与运营部）；Columbia University（哥伦比亚大学）；Tsinghua University（清华大学）
By generating prediction intervals (PIs) to quantify the uncertainty of each prediction in deep learning regression, the risk of wrong predictions can be effectively controlled. High-quality PIs need to be as narrow as possible, whilst covering a preset proportion of real labels. At present, many approaches to improve the quality of PIs can effectively reduce the width of PIs, but they do not ensure that enough real labels are captured. Inductive Conformal Predictor (ICP) is an algorithm that can generate effective PIs which is theoretically guaranteed to cover a preset proportion of data. How…
Prevailing methods for graphs require abundant label and edge information for learning. When data for a new task are scarce, meta-learning can learn from prior experiences and form much-needed inductive biases for fast adaption to new tasks. Here, we introduce G-Meta, a novel meta-learning algorithm for graphs. G-Meta uses local subgraphs to transfer subgraph-specific information and learn transferable knowledge faster via meta gradients. G-Meta learns how to quickly adapt to a new task using only a handful of nodes or edges in the new task and does so by learning from data points in other gra…
In a data-scarce field such as healthcare, where models often deliver predictions on patients with rare conditions, the ability to measure the uncertainty of a model's prediction could potentially lead to improved effectiveness of decision support tools and increased user trust. This work advances the understanding of uncertainty estimation for classification and risk prediction on medical tabular data, in a two-fold way. First, we expand and refine the set of heuristics to select an uncertainty estimation technique, introducing tests for clinically-relevant scenarios such as generalization to…
Inductive inference of gradient-boosted decision trees on graphs for insurance fraud detection
Data Office, Allianz Benelux（Allianz Benelux 数据办公室）；KU Leuven University of Antwerp-imec（安特卫普-imec 联合大学KU Leuven）；KU Leuven；University of Antwerp-imec（安特卫普-imec 大学）
RheinMain University of Applied Sciences（莱茵美因应用科学大学）
We address the challenge of building domain-specific knowledge models for industrial use cases, where labelled data and taxonomic information is initially scarce. Our focus is on inductive link prediction models as a basis for practical tools that support knowledge engineers with exploring text collections and discovering and linking new (so-called open-world) entities to the knowledge graph. We argue that - though neural approaches to text mining have yielded impressive results in the past years - current benchmarks do not reflect the typical challenges encountered in the industrial wild prop…
Causal Data Augmentation for Robust Fine-Tuning of Tabular Foundation Models
Department of Computer Science University of Freiburg（计算机科学系弗赖堡大学）；University of Freiburg（弗赖堡大学）；Prior Labs ELLIS Institute Tübingen University of Freiburg（Prior实验室ELLIS研究所弗赖堡大学）
