arXiv 1509.024382015-09-08stat.ML

A Variational Bayesian State-Space Approach to Online Passive-Aggressive Regression

Arnold Salas, Stephen J. Roberts, Michael A. Osborne

英文摘要：

Online Passive-Aggressive (PA) learning is a class of online margin-based algorithms suitable for a wide range of real-time prediction tasks, including classification and regression. PA algorithms are formulated in terms of deterministic point-estimation problems governed by a set of user-defined hyperparameters: the approach fails to capture model/prediction uncertainty and makes their performance highly sensitive to hyperparameter configurations. In this paper, we introduce a novel PA learning framework for regression that overcomes the above limitations. We contribute a Bayesian state-space interpretation of PA regression, along with a novel online variational inference scheme, that not only produces probabilistic predictions, but also offers the benefit of automatic hyperparameter tuning. Experiments with various real-world data sets show that our approach performs significantly better than a more standard, linear Gaussian state-space model.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

阅读原文

阅读 PDF ↗

arXiv 页面 ↗HTML 原文 ↗

本文内容

英文摘要相关论文

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

继续探索

相关论文

Online Passive-Aggressive Total-Error-Rate Minimization

National University of Singapore（新加坡国立大学）；Department of Statistics and Applied Probability, National University of Singapore（新加坡国立大学统计与应用概率系）

We provide a new online learning algorithm which utilizes online passive-aggressive learning (PA) and total-error-rate minimization (TER) for binary classification. The PA learning establishes not only large margin training but also the capacity to handle non-separable data. The TER learning on the other hand minimizes an approximated classification error based objective function. We propose an online PATER algorithm which combines those useful properties. In addition, we also present a weighted PATER algorithm to improve the ability to cope with data imbalance problems. Experimental results d…

2020-02-05cs.LGstat.ML

Exact Passive-Aggressive Algorithms for Learning to Rank Using Interval Labels

IIIT Hyderabad（海得拉巴国际信息技术学院）

In this paper, we propose exact passive-aggressive (PA) online algorithms for learning to rank. The proposed algorithms can be used even when we have interval labels instead of actual labels for examples. The proposed algorithms solve a convex optimization problem at every trial. We find exact solution to those optimization problems to determine the updated parameters. We propose support class algorithm (SCA) which finds the active constraints using the KKT conditions of the optimization problems. These active constrains form support set which determines the set of thresholds that need to be u…

2018-08-18cs.LGstat.ML

基于加权水库采样的线性被动-主动在线学习稳定方法

Stabilizing Linear Passive-Aggressive Online Learning with Weighted Reservoir Sampling

Booz Allen Hamilton（博思艾伦汉密尔顿公司）；Harvard University（哈佛大学）；Stanford University（斯坦福大学）；University of Maryland, Baltimore County（马里兰大学巴尔的摩县分校）；Laboratory for Physical Sciences（物理科学实验室）

针对在线学习算法易受离群点影响导致最终解准确率低的问题，提出基于加权水库采样的集成方法，利用被动轮次估计解的质量，在两类在线学习算法上取得显著提升且风险有界。

2024-10-31cs.LGstat.ML

Lagging Inference Networks and Posterior Collapse in Variational Autoencoders

Carnegie Mellon University（卡内基梅隆大学）；University of California San Diego（加利福尼亚大学圣地亚哥分校）

The variational autoencoder (VAE) is a popular combination of deep latent variable model and accompanying variational learning technique. By using a neural inference network to approximate the model's posterior on latent variables, VAEs efficiently parameterize a lower bound on marginal data likelihood that can be optimized directly via gradient methods. In practice, however, VAE training often results in a degenerate local optimum known as "posterior collapse" where the model learns to ignore the latent variable and the approximate posterior mimics the prior. In this paper, we investigate pos…

2019-01-28cs.LGstat.ML

Large Scale Local Online Similarity/Distance Learning Framework based on Passive/Aggressive

Ferdowsi University of Mashhad（菲尔多西马什哈德大学）

Similarity/Distance measures play a key role in many machine learning, pattern recognition, and data mining algorithms, which leads to the emergence of metric learning field. Many metric learning algorithms learn a global distance function from data that satisfy the constraints of the problem. However, in many real-world datasets that the discrimination power of features varies in the different regions of input space, a global metric is often unable to capture the complexity of the task. To address this challenge, local metric learning methods are proposed that learn multiple metrics across th…

2018-04-05cs.CVcs.LGstat.ML

Bayesian Robust Tensor Ring Model for Incomplete Multiway Data

Guangdong University of Technology（广东工业大学）；RIKEN（理化学研究所）

Robust tensor completion (RTC) aims to recover a low-rank tensor from its incomplete observation with outlier corruption. The recently proposed tensor ring (TR) model has demonstrated superiority in solving the RTC problem. However, the existing methods either require a pre-assigned TR rank or aggressively pursue the minimum TR rank, thereby often leading to biased solutions in the presence of noise. In this paper, a Bayesian robust tensor ring decomposition (BRTR) method is proposed to give more accurate solutions to the RTC problem, which can avoid exquisite selection of the TR rank and pena…

2023-02-15cs.AIcs.LGstat.ML

更多相关论文收起更多论文

更加主动！理解变分自编码器均值表示与采样表示之间的差异

Be More Active! Understanding the Differences between Mean and Sampled Representations of Variational Autoencoders

University of Kent（肯特大学）

本文通过选择性后验坍缩视角，发现变分自编码器的均值表示与采样表示差异源于被动变量，但均值表示仍适用于下游任务，移除被动变量可能有益。

2023-12-25cs.LGstat.ML

Cumulative Sum Ranking

Pontifícia Univesidade Católica do Rio de Janeiro（里约热内卢天主教大学）

The goal of Ordinal Regression is to find a rule that ranks items from a given set. Several learning algorithms to solve this prediction problem build an ensemble of binary classifiers. Ranking by Projecting uses interdependent binary perceptrons. These perceptrons share the same direction vector, but use different bias values. Similar approaches use independent direction vectors and biases. To combine the binary predictions, most of them adopt a simple counting heuristics. Here, we introduce a novel cumulative sum scoring function to combine the binary predictions. The proposed score value ag…

2019-11-25cs.LGstat.ML

Low-Rank Robust Online Distance/Similarity Learning based on the Rescaled Hinge Loss

Sabzevar University of New Technology（萨布泽瓦尔新技术大学）；Ferdowsi University of Mashhad（马什哈德费尔多西大学）；Hakim Sabzevari University（哈基姆萨布泽瓦里大学）

An important challenge in metric learning is scalability to both size and dimension of input data. Online metric learning algorithms are proposed to address this challenge. Existing methods are commonly based on (Passive Aggressive) PA approach. Hence, they can rapidly process large volumes of data with an adaptive learning rate. However, these algorithms are based on the Hinge loss and so are not robust against outliers and label noise. Also, existing online methods usually assume training triplets or pairwise constraints are exist in advance. However, many datasets in real-world applications…

2020-10-10cs.CVcs.LGstat.ML