首页/论文/2004.12293

arXiv 2004.122932020-04stat.MEmath.STstat.MLstat.TH

Classification Trees for Imbalanced and Sparse Data: Surface-to-Volume Regularization

Yichen Zhu, Cheng Li, David B. Dunson

备注：: Submitted to Journal of American Statistical Association

英文摘要：

Classification algorithms face difficulties when one or more classes have limited training data. We are particularly interested in classification trees, due to their interpretability and flexibility. When data are limited in one or more of the classes, the estimated decision boundaries are often irregularly shaped due to the limited sample size, leading to poor generalization error. We propose a novel approach that penalizes the Surface-to-Volume Ratio (SVR) of the decision set, obtaining a new class of SVR-Tree algorithms. We develop a simple and computationally efficient implementation while proving estimation consistency for SVR-Tree and rate of convergence for an idealized empirical risk minimizer of SVR-Tree. SVR-Tree is compared with multiple algorithms that are designed to deal with imbalance through real data applications.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

阅读原文

阅读 PDF ↗

arXiv 页面 ↗HTML 原文 ↗

本文内容

英文摘要相关论文

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

继续探索

相关论文

Adaptive Ensemble of Classifiers with Regularization for Imbalanced Data Classification

Sichuan University（四川大学）；Rutgers University – New Brunswick（罗格斯大学新不伦瑞克分校）；Zhejiang University（浙江大学）；Tennessee State University（田纳西州立大学）

The dynamic ensemble selection of classifiers is an effective approach for processing label-imbalanced data classifications. However, such a technique is prone to overfitting, owing to the lack of regularization methods and the dependence of the aforementioned technique on local geometry. In this study, focusing on binary imbalanced data classification, a novel dynamic ensemble method, namely adaptive ensemble of classifiers with regularization (AER), is proposed, to overcome the stated limitations. The method solves the overfitting problem through implicit regularization. Specifically, it lev…

2020-11-06cs.LGstat.ML

Class-Imbalanced Semi-Supervised Learning

Seoul National University（首尔国立大学）；SK hynix（SK海力士）

Semi-Supervised Learning (SSL) has achieved great success in overcoming the difficulties of labeling and making full use of unlabeled data. However, SSL has a limited assumption that the numbers of samples in different classes are balanced, and many SSL algorithms show lower performance for the datasets with the imbalanced class distribution. In this paper, we introduce a task of class-imbalanced semi-supervised learning (CISSL), which refers to semi-supervised learning with class-imbalanced data. In doing so, we consider class imbalance in both labeled and unlabeled sets. First, we analyze ex…

2020-02-17cs.CVcs.LGstat.ML

ADMM Algorithmic Regularization Paths for Sparse Statistical Machine Learning

Optimization approaches based on operator splitting are becoming popular for solving sparsity regularized statistical machine learning models. While many have proposed fast algorithms to solve these problems for a single regularization parameter, conspicuously less attention has been given to computing regularization paths, or solving the optimization problems over the full range of regularization parameters to obtain a sequence of sparse models. In this chapter, we aim to quickly approximate the sequence of sparse models associated with regularization paths for the purposes of statistical mod…

2015-04-24stat.ME

Fast Learning Rate of Non-Sparse Multiple Kernel Learning and Optimal Regularization Strategies

In this paper, we give a new generalization error bound of Multiple Kernel Learning (MKL) for a general class of regularizations, and discuss what kind of regularization gives a favorable predictive accuracy. Our main target in this paper is dense type regularizations including \ellp-MKL. According to the recent numerical experiments, the sparse regularization does not necessarily show a good performance compared with dense type regularizations. Motivated by this fact, this paper gives a general theoretical tool to derive fast learning rates of MKL that is applicable to arbitrary mixed-norm-ty…

2011-11-16stat.ML

PET-TURTLE：深度无监督支持向量机用于不平衡数据簇

PET-TURTLE: Deep Unsupervised Support Vector Machines for Imbalanced Data Clusters

University of Michigan（密歇根大学）

PET-TURTLE通过引入幂律先验和稀疏logits，改进了传统TURTLE在处理不平衡数据时的聚类性能。

2026-01-06cs.LGeess.IVstat.ML

在稀疏和不平衡情形下确定社区数量

Determining the Number of Communities in Sparse and Imbalanced Settings

本文针对稀疏和不平衡网络中社区数量K难以估计的问题，提出一种基于中心化非回溯算子的谱方法，利用最大特征值进行拟合优度检验，有效克服弱信号和度数不平衡挑战。

2024-06-06cs.SIphysics.soc-phstat.ME

更多相关论文收起更多论文

Efficient algorithm to select tuning parameters in sparse regression modeling with regularization

In sparse regression modeling via regularization such as the lasso, it is important to select appropriate values of tuning parameters including regularization parameters. The choice of tuning parameters can be viewed as a model selection and evaluation problem. Mallows' $C_p$ type criteria may be used as a tuning parameter selection tool in lasso-type regularization methods, for which the concept of degrees of freedom plays a key role. In the present paper, we propose an efficient algorithm that computes the degrees of freedom by extending the generalized path seeking algorithm. Our procedure …

2012-01-04stat.COstat.MEstat.ML

Feature Grouping and Sparse Principal Component Analysis with Truncated Regularization

In this paper, we consider a new variant for principal component analysis (PCA), aiming to capture the grouping and/or sparse structures of factor loadings simultaneously. To achieve these goals, we employ a non-convex truncated regularization with naturally adjustable sparsity and grouping effects, and propose the Feature Grouping and Sparse Principal Component Analysis (FGSPCA). The proposed FGSPCA method encourages the factor loadings with similar values to collapse into disjoint homogeneous groups for feature grouping or into a special zero-valued group for feature selection, which in turn…

2022-09-13stat.MEstat.ML

SparseStep: Approximating the Counting Norm for Sparse Regularization

The SparseStep algorithm is presented for the estimation of a sparse parameter vector in the linear regression problem. The algorithm works by adding an approximation of the exact counting norm as a constraint on the model parameters and iteratively strengthening this approximation to arrive at a sparse solution. Theoretical analysis of the penalty function shows that the estimator yields unbiased estimates of the parameter vector. An iterative majorization algorithm is derived which has a straightforward implementation reminiscent of ridge regression. In addition, the SparseStep algorithm is …

2017-01-24stat.ME

基于稀疏正则化的整合分析中的多元回归建模

Multivariate regression modeling in integrative analysis via sparse regularization

本文提出一种基于稀疏正则化的整合分析多元回归建模方法，利用交替方向乘子法实现变量与组选择，并通过模拟和污水处理微生物数据验证其优于单一数据集分析。

2023-04-15stat.MEstat.ML

Sparse Unit-Sum Regression

This paper considers sparsity in linear regression under the restriction that the regression weights sum to one. We propose an approach that combines $\ell_0$- and $\ell_1$-regularization. We compute its solution by adapting a recent methodological innovation made by Bertsimas et al. (2016) for $\ell_0$-regularization in standard linear regression. In a simulation experiment we compare our approach to $\ell_0$-regularization and $\ell_1$-regularization and find that it performs favorably in terms of predictive performance and sparsity. In an application to index tracking we show that our appro…

2019-07-10stat.COstat.ME

Sparse Reduced Rank Regression With Nonconvex Regularization

The Hong Kong University of Science and Technology（香港科技大学）

In this paper, the estimation problem for sparse reduced rank regression (SRRR) model is considered. The SRRR model is widely used for dimension reduction and variable selection with applications in signal processing, econometrics, etc. The problem is formulated to minimize the least squares loss with a sparsity-inducing penalty considering an orthogonality constraint. Convex sparsity-inducing functions have been used for SRRR in literature. In this work, a nonconvex function is proposed for better sparsity inducing. An efficient algorithm is developed based on the alternating minimization (or…

2018-03-20cs.LGq-fin.CPstat.ME

超越正则化：固有稀疏主成分分析

Beyond Regularization: Inherently Sparse Principal Component Analysis

针对正则化稀疏 PCA 偏离总体结构且奇异向量不正交的问题，本文利用协方差矩阵的稀疏块对角结构，提出构造固有稀疏且正交奇异向量的方法，并通过模拟与真实数据验证其有效性。

2025-10-04stat.ME