17 pages, 5 figures, 3 tables, 2 algorithms, authors' version of paper accepted in PAKDD 2026 special session on Data Science: Foundations and Applications (DSFA)
Many real-world applications generate continuous data streams for regression. Hoeffding trees and their variants have a long-standing tradition due to their effectiveness, either alone or as base models in broader ensembles. Recent batch-learning work shows that kernel density estimation (KDE) improves smoothed predictions in imbalanced regression [Yang et al., 2021], while hierarchical shrinkage (HS) provides post-hoc regularization for decision trees without modifying their structure [Agarwal et al., 2022]. We extend KDE to streaming settings via a telescoping formulation and integrate HS into incremental decision trees. Empirical evaluation on standard online regression benchmarks shows that KDE consistently improves early-stream performance, whereas HS provides limited gains. Our implementation is publicly available at: https://github.com/marinaAlchirch/DSFA_2026.
Highly Imbalanced Regression with Tabular Data in SEP and Other Applications
Department of Electrical Engineering and Computer Science（电气工程与计算机科学系）；Department of Aerospace, Physics and Space Sciences（航空航天、物理与空间科学系）；Florida Institute of Technology（佛罗里达理工学院）
MIT Computer Science & Artificial Intelligence Laboratory（麻省理工学院计算机科学与人工智能实验室）；Department of Computer Science, Rutgers University（罗格斯大学计算机科学系）
Real-world data often exhibit imbalanced distributions, where certain target values have significantly fewer observations. Existing techniques for dealing with imbalanced data focus on targets with categorical indices, i.e., different classes. However, many tasks involve continuous targets, where hard boundaries between classes do not exist. We define Deep Imbalanced Regression (DIR) as learning from such imbalanced data with continuous targets, dealing with potential missing data for certain target values, and generalizing to the entire target range. Motivated by the intrinsic difference betw…
Simon Fraser University（西蒙弗雷泽大学）；Borealis AI（Borealis人工智能研究院）
Data imbalance, in which a plurality of the data samples come from a small proportion of labels, poses a challenge in training deep neural networks. Unlike classification, in regression the labels are continuous, potentially boundless, and form a natural ordering. These distinct features of regression call for new techniques that leverage the additional information encoded in label-space relationships. This paper presents the RankSim (ranking similarity) regularizer for deep imbalanced regression, which encodes an inductive bias that samples that are closer in label space should also be closer…
Faculty of Sciences - University of Porto（波尔图大学理学院）；INESC TEC（葡萄牙国家电气与计算机工程系统研究所）
Imbalanced domain learning aims to produce accurate models in predicting instances that, though underrepresented, are of utmost importance for the domain. Research in this field has been mainly focused on classification tasks. Comparatively, the number of studies carried out in the context of regression tasks is negligible. One of the main reasons for this is the lack of loss functions capable of focusing on minimizing the errors of extreme (rare) values. Recently, an evaluation metric was introduced: Squared Error Relevance Area (SERA). This metric posits a bigger emphasis on the errors commi…
Dist Loss: Enhancing Regression in Few-Shot Region through Distribution Distance Constraint
Institute of Medical Technology, Health Science Center of Peking University（北京大学健康科学中心医学技术研究所）；National Institute of Health Data Science, Peking University（北京大学国家健康数据科学研究院）；Institute for Artificial Intelligence, Peking University（北京大学人工智能研究院）
We consider the problem of mixed linear regression (MLR), where each observed sample belongs to one of $K$ unknown linear models. In practical applications, the proportions of the $K$ components are often imbalanced. Unfortunately, most MLR methods do not perform well in such settings. Motivated by this practical challenge, in this work we propose Mix-IRLS, a novel, simple and fast algorithm for MLR with excellent performance on both balanced and imbalanced mixtures. In contrast to popular approaches that recover the $K$ models simultaneously, Mix-IRLS does it sequentially using tools from rob…
In this work, we consider the problem of imbalanced data in a regression framework when the imbalanced phenomenon concerns continuous or discrete covariates. Such a situation can lead to biases in the estimates. In this case, we propose a data augmentation algorithm that combines a weighted resampling (WR) and a data augmentation (DA) procedure. In a first step, the DA procedure permits exploring a wider support than the initial one. In a second step, the WR method drives the exogenous distribution to a target one. We discuss the choice of the DA procedure through a numerical study that illust…
Comparative Analysis of Data Preprocessing Methods, Feature Selection Techniques and Machine Learning Models for Improved Classification and Regression Performance on Imbalanced Genetic Data
Evergreen Valley College（埃弗格林谷学院）；Gilead Sciences（吉利德科学公司）
Qingdao University（青岛大学）；University of Kentucky（肯塔基大学）
We introduce a discriminative regression approach to supervised classification in this paper. It estimates a representation model while accounting for discriminativeness between classes, thereby enabling accurate derivation of categorical information. This new type of regression models extends existing models such as ridge, lasso, and group lasso through explicitly incorporating discriminative information. As a special case we focus on a quadratic model that admits a closed-form analytical solution. The corresponding classifier is called discriminative regression machine (DRM). Three iterative…
Beyond Homoscedasticity: Decoupled Uncertainty Optimization for Deep Imbalanced Regression
School of Cyber Science and Engineering, Wuhan University（武汉大学网络空间安全学院）；School of Mathematics and Statistics, Wuhan University（武汉大学数学与统计学院）；School of Synthetic Biology and Biomanufacturing, Tianjin University（天津大学合成生物学与生物制造学院）
SPECTRA: Spectral Domain-Aware Graph Generation for Imbalanced Molecular Property Regression
University of Notre Dame, Dept. of Computer Science；University of Notre Dame, Dept. of Chemistry；University of Notre Dame, Lucy Family Institute for Data \& Society Notre Dame Indiana USA；University of Notre Dame, Lucy Family Institute for Data \& Society
Deep Imbalanced Regression to Estimate Vascular Age from PPG Data: a Novel Digital Biomarker for Cardiovascular Health
Institute of Medical Technology, Health Science Center of Peking University（北京大学健康科学中心医学技术研究所）；National Institute of Health Data Science, Peking University（北京大学国家健康数据科学研究院）；Peking University People’s Hospital（北京大学人民医院）；College of Electronic Science and Engineering, Jilin University（吉林大学电子科学与工程学院）
针对 PPG 血管年龄估计中的年龄不平衡问题，提出 Dist Loss 并训练 Net1D，在 UK Biobank 上取得最优表现，且预测血管年龄与长期心血管事件相关。
In many real-world pattern recognition scenarios, such as in medical applications, the corresponding classification tasks can be of an imbalanced nature. In the current study, we focus on binary, imbalanced classification tasks, i.e.~binary classification tasks in which one of the two classes is under-represented (minority class) in comparison to the other class (majority class). In the literature, many different approaches have been proposed, such as under- or oversampling, to counter class imbalance. In the current work, we introduce a novel method, which addresses the issues of class imbala…
PARIS: Pruning Algorithm via the Representer theorem for Imbalanced Scenarios
Department of Physics and Astronomy（物理与天文学系）；Queen Mary University of London（伦敦女王玛丽大学）；University of Colorado（科罗拉多大学）；SWx-TREC (Space Weather Technology Research and Education Center)（空间天气技术研究与教育中心）
University of São Paulo（圣保罗大学）；State University of Londrina（隆德里纳州立大学）
One of the current challenges in machine learning is how to deal with data coming at increasing rates in data streams. New predictive learning strategies are needed to cope with the high throughput data and concept drift. One of the data stream mining tasks where new learning strategies are needed is multi-target regression, due to its applicability in a high number of real world problems. While reliable and effective learning strategies have been proposed for batch multi-target regression, few have been proposed for multi-target online learning in data streams. Besides, most of the existing s…
Nowadays with a growing number of online controlling systems in the organization and also a high demand of monitoring and stats facilities that uses data streams to log and control their subsystems, data stream mining becomes more and more vital. Hoeffding Trees (also called Very Fast Decision Trees a.k.a. VFDT) as a Big Data approach in dealing with the data stream for classification and regression problems showed good performance in handling facing challenges and making the possibility of any-time prediction. Although these methods outperform other methods e.g. Artificial Neural Networks (AN…
Data imbalance is common in production data, where controlled production settings require data to fall within a narrow range of variation and data are collected with quality assessment in mind, rather than data analytic insights. This imbalance negatively impacts the predictive performance of models on underrepresented observations. We propose sampling to adjust for this imbalance with the goal of improving the performance of models trained on historical production data. We investigate the use of three sampling approaches to adjust for imbalance. The goal is to downsample the covariates in the…
Bayesian Regression for Predicting Subscription to Bank Term Deposits in Direct Marketing Campaigns
Institute of Computer Science and Computational Science, Universität Potsdam（波茨坦大学计算机科学与计算科学研究所）；Faculty of Statistics, Technische Universität Dortmund（多特蒙德工业大学统计学院）
Institute of Mathematics and Computer Sciences（数学与计算机科学研究所）；University of São Paulo（圣保罗大学）
A central aspect of online decision tree solutions is evaluating the incoming data and enabling model growth. For such, trees much deal with different kinds of input features and partition them to learn from the data. Numerical features are no exception, and they pose additional challenges compared to other kinds of features, as there is no trivial strategy to choose the best point to make a split decision. The problem is even more challenging in regression tasks because both the features and the target are continuous. Typical online solutions evaluate and store all the points monitored betwee…
