Entropy inference and the James-Stein estimator, with application to nonlinear gene association networks

JSRT: James-Stein Regression Tree

Regression tree (RT) has been widely used in machine learning and data mining community. Given a target data for prediction, a regression tree is first constructed based on a training dataset before making prediction for each leaf node. In practice, the performance of RT relies heavily on the local mean of samples from an individual node during the tree construction/prediction stage, while neglecting the global information from different nodes, which also plays an important role. To address this issue, we propose a novel regression tree, named James-Stein Regression Tree (JSRT) by considering …

使用 James-Stein 估计量的黑盒变分推断方差控制

Variance Control for Black Box Variational Inference Using The James-Stein Estimator

本文针对黑盒变分推断不稳定且需调参的问题，将随机梯度上升重构为多元估计，并用 James-Stein 估计量控制梯度方差，实现免调参且性能相当或更优。

Cluster-Seeking James-Stein Estimators

This paper considers the problem of estimating a high-dimensional vector of parameters $\boldsymbolθ \in \mathbb{R}^n$ from a noisy observation. The noise vector is i.i.d. Gaussian with known variance. For a squared-error loss function, the James-Stein (JS) estimator is known to dominate the simple maximum-likelihood (ML) estimator when the dimension $n$ exceeds two. The JS-estimator shrinks the observed vector towards the origin, and the risk reduction over the ML-estimator is greatest for $\boldsymbolθ$ that lie close to the origin. JS-estimators can be generalized to shrink the data towards…

批量归一化在存在对抗攻击时的Stein收缩可接受性

Admissibility of Stein Shrinkage for Batch Normalization in the Presence of Adversarial Attacks

本文研究了在存在对抗攻击时，Stein收缩在批量归一化中的可接受性，并展示了其在图像分类和分割任务中的应用效果。

Multi-Task Averaging

We present a multi-task learning approach to jointly estimate the means of multiple independent data sets. The proposed multi-task averaging (MTA) algorithm results in a convex combination of the single-task maximum likelihood estimates. We derive the optimal minimum risk estimator and the minimax estimator, and show that these estimators can be efficiently estimated. Simulations and real data experiments demonstrate that MTA estimators often outperform both single-task and James-Stein estimators.

C-SURE: Shrinkage Estimator and Prototype Classifier for Complex-Valued Deep Learning

The James-Stein (JS) shrinkage estimator is a biased estimator that captures the mean of Gaussian random vectors.While it has a desirable statistical property of dominance over the maximum likelihood estimator (MLE) in terms of mean squared error (MSE), not much progress has been made on extending the estimator onto manifold-valued data. We propose C-SURE, a novel Stein's unbiased risk estimate (SURE) of the JS estimator on the manifold of complex-valued data with a theoretically proven optimum over MLE. Adapting the architecture of the complex-valued SurReal classifier, we further incorporate…