We consider the problem of estimation and structure learning of high dimensional signals via a normal sequence model, where the underlying parameter vector is piecewise constant, or has a block structure. We develop a Bayesian fusion estimation method by using the Horseshoe prior to induce a strong shrinkage effect on successive differences in the mean parameters, simultaneously imposing sufficient prior concentration for non-zero values of the same. The proposed method thus facilitates consistent estimation and structure recovery of the signal pieces. We provide theoretical justifications of our approach by deriving posterior convergence rates and establishing selection consistency under suitable assumptions. We also extend our proposed method to signal de-noising over arbitrary graphs and develop efficient computational methods along with providing theoretical guarantees. We demonstrate the superior performance of the Horseshoe based Bayesian fusion estimation method through extensive simulations and two real-life examples on signal de-noising in biological and geophysical applications. We also demonstrate the estimation performance of our method on a real-world large network for the graph signal de-noising problem.
Dirichlet Process Mixture Models with Shrinkage Prior
我们提出了Dirichlet过程混合（Dirichlet Process Mixture）模型的预测和集群明智的变量选择，基于两种选择的收缩基线先验分布的线性回归系数，即马蹄先验和正态伽玛先验。我们在模拟研究中表明，这两个建议的ESTA模型往往优于标准ESTA模型的基础上的非收缩正常的先验，在预测，变量选择和聚类精度。这对于马蹄模型尤其如此，并且当协变量的数量超过群内样本量时。一个真实的数据集进行了分析，以说明所提出的建模方法，两个建议的预测模型再次获得更好的预测精度。
Graduate School of Economics, The University of Tokyo（东京大学经济学研究科）；Center for Spatial Information Science, The University of Tokyo（东京大学空间信息科学中心）
Due to developments in instruments and computers, functional observations are increasingly popular. However, effective methodologies for flexibly estimating the underlying trends with valid uncertainty quantification for a sequence of functional data (e.g. functional time series) are still scarce. In this work, we develop a locally adaptive smoothing method, called functional horseshoe smoothing, by introducing a shrinkage prior to the general order of differences of functional variables. This allows us to capture abrupt changes by making the most of the shrinkage capability and also to assess…
Horseshoe Regularization for Machine Learning in Complex and Deep Models
自从正则化的马蹄先验出现以来，全局-局部收缩方法已被证明是机器学习中贝叶斯方法发展的沃土，特别是对于高维回归和分类问题。它们在计算上取得了显著的成功，并得到了强有力的理论支持。大多数现有文献都集中在线性高斯情况下;参见Bhadra et al.（2019 b）进行系统调查。本文的目的是通过回顾与机器学习应用更相关的复杂模型中的方法和计算发展，证明马蹄正则化的有用性。具体来说，我们专注于非线性和非高斯模型中的马蹄正则化方法的挑战;多变量模型;和深度神经网络。我们还概述了最近的计算马蹄形收缩的复杂模型的发展，以及可用的软件实现列表，允许一个冒险超出舒适区的典型线性回归问题。
Posterior Concentration Properties of a General Class of Shrinkage Priors around Nearly Black Vectors
假设我们有一个根据多元正态分布生成的数据，其中有一个固定的未知均值向量，该向量在几乎为黑色的意义上是稀疏的。最优性的贝叶斯估计和后验浓度性质的极小极大风险的$l_2$规范对应于一个非常普遍的类连续收缩先验研究在这项工作中。考虑的先验类是丰富的，足以包括各种各样的重尾先验分布，如，三参数β正态混合（包括马蹄铁），广义双帕累托，逆伽玛和正常指数伽玛先验。假设非零分量的平均向量的数量是已知的，我们表明，贝叶斯估计对应于这一一般类先验达到极小极大风险的$l_2$规范（可能高达一个乘法常数）和相应的后验分布合同周围的真实均值向量在极小极大最优率适当选择的全球收缩参数。此外，我们提供的条件，这些后验分布合同周围相应的贝叶斯估计至少一样快的极小极大风险的$l_2$规范。我们还提供了一个下界的总后验方差的一个重要子类，这一般类收缩先验，包括广义双帕累托先验的形状参数$α=0.5$和三个参数β正常混合物的参数$a=0.5$和$b>0$（包括马蹄铁），特别是。本工作的灵感来自van der Pas等人（2014）最近的工作。在本设置下，马蹄先验的后收缩特性。我们扩展了他们的结果，这类先验知识，并提出了新的统一的证明工作的一个非常广泛的一类一组连续收缩先验。
In this paper, we focus on identifying differentially activated brain regions using a light sheet fluorescence microscopy - a recently developed technique for whole-brain imaging. Most existing statistical methods solve this problem by partitioning the brain regions into two classes: significantly and non-significantly activated. However, for the brain imaging problem at the center of our study, such binary grouping may provide overly simplistic discoveries by filtering out weak but important signals, that are typically adulterated by the noise present in the data. To overcome this limitation,…
Achieving Parsimony in Bayesian VARs with the Horseshoe Prior
在向量自回归（VAR）模型或任何多变量回归模型的上下文中，相关预测因子的数量相对于从其构建预测方程的可用信息集可能较小。众所周知，基于（未惩罚的）最小二乘估计的预测可能会过拟合数据，导致预测结果不佳。自从明尼苏达先验被提出以来（Doan et al.（1984）），已经开发了许多旨在提高预测性能的方法。在本文中，我们在贝叶斯VAR的背景下提出了马蹄先验（Carvalho et al.（2010），Carvalho et al.（2009））。马蹄先验是一种独特的收缩先验方案，它将不相关的信号严格地收缩到0，同时允许大信号保持大并且实际上未收缩。在一项实证研究中，我们表明，马蹄事先竞争有利的收缩方案中常用的贝叶斯VAR模型，以及与前，在系数向量中施加真正的稀疏性。此外，我们建议使用粒子吉布斯与向后模拟（Lindsten et al.（2012），Andrieu et al.（2010））的估计随时间变化的波动性参数。我们提供了在线补充材料中使用的所有MCMC方法的详细描述。
In many large-scale inverse problems, such as computed tomography and image deblurring, characterization of sharp edges in the solution is desired. Within the Bayesian approach to inverse problems, edge-preservation is often achieved using Markov random field priors based on heavy-tailed distributions. Another strategy, popular in statistics, is the application of hierarchical shrinkage priors. An advantage of this formulation lies in expressing the prior as a conditionally Gaussian distribution depending of global and local hyperparameters which are endowed with heavy-tailed hyperpriors. In t…
We develop a novel full-Bayesian approach for multiple correlated precision matrices, called multiple Graphical Horseshoe (mGHS). The proposed approach relies on a novel multivariate shrinkage prior based on the Horseshoe prior that borrows strength and shares sparsity patterns across groups, improving posterior edge selection when the precision matrices are similar. On the other hand, there is no loss of performance when the groups are independent. Moreover, mGHS provides a similarity matrix estimate, useful for understanding network similarities across groups. We implement an efficient Metro…
Spike-and-slab shrinkage priors for structurally sparse Bayesian neural networks
Computational Science Initiative, Brookhaven National Laboratory（布鲁克海文国家实验室计算科学倡议）；Department of Statistics and Probability, Michigan State University（密歇根州立大学统计与概率系）
Booth School of Business University of Chicago（芝加哥大学商学院）；Volgenau School of Engineering, George Mason University（乔治·马歇尔大学工程学院）；Department of Systems Engineering and Operations Research George Mason University（乔治·马歇尔大学系统工程与运营管理系）
Learning Compositional Sparse Gaussian Processes with a Shrinkage Prior
Ulsan National Institute of Science and Technology（蔚山国立科学技术院）；VinAI Research（VinAI研究院）；Korea Advanced Institute of Science and Technology（韩国科学技术院）；INEEJI
Bayesian Transfer Learning for Enhanced Estimation and Inference
Vagelos College of Physicians and Surgeons, Columbia University（哥伦比亚大学瓦格洛斯医师与外科医学院）；Columbia University Mailman School of Public Health（哥伦比亚大学米尔曼公共卫生学院）；University of California, Los Angeles（加州大学洛杉矶分校）；The University of Texas MD Anderson Cancer Center（德克萨斯大学安德森癌症中心）
