发表机构
The Institute of Statistical Mathematics（统计科学研究所）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
本文提出一种协变量依赖的非参数 $g$-建模回归框架，通过无限混合专家类实现 NPMLE，自动确定专家数量，并证明其 Lipschitz 连续性与门函数逼近能力，在合成和真实数据上验证了有效性。
AI 中文摘要
经验贝叶斯 $g$-建模通过从观测数据中估计潜在先验分布来捕捉单元级异质性。然而，在现有公式中，先验是所有单元共享的。在本文中，我们开发了一种用于回归的协变量依赖的 $g$-建模框架，其中回归系数的整个先验分布被允许依赖于协变量。我们将先验的估计表述为协变量依赖先验的非参数最大似然估计（NPMLE），并表明无约束问题是不适定的。为解决此问题，我们引入了条件先验的无限混合专家类，在该类下，NPMLE 恰好是一个 softmax 门控的混合专家（MoE），其专家数量不是预先固定的，而是由数据决定。基于一阶最优性条件，我们提出了两种基于样本的估计算法，自动选择专家，并辅以事后聚合专家以进行解释。在理论方面，我们证明了该类中的每个条件先验在协变量上是 Lipschitz 连续的，并且聚合的 softmax 门可以近似任何连续的门函数。通过应用于合成数据集和真实数据集，展示了所提出的 NPMLE 的有效性。
英文摘要
Empirical Bayes $g$-modeling captures unit-level heterogeneity by estimating a latent prior distribution from observed data. In the existing formulations, however, the prior is shared by all units. In this paper, we develop a covariate-dependent g-modeling framework for regression in which the entire prior distribution of the regression coefficients is allowed to depend on covariates. We formulate the estimation of the prior as nonparametric maximum likelihood estimation (NPMLE) of the covariate-dependent prior, and show that the unrestricted problem is ill-posed. To resolve this, we introduce the infinite Mixture-of-Expertizing class of conditional priors, under which the NPMLE is precisely a softmax-gated Mixture of Experts (MoE) whose number of experts is not fixed in advance but is determined by the data. Building on a first-order optimality condition, we propose two exemplar-based estimation algorithms that select experts automatically, together with a post-hoc aggregation of experts for interpretation. On the theoretical side, we show that every conditional prior in the class is Lipschitz continuous in the covariates, and that aggregated softmax gates can approximate any continuous gate function. The effectiveness of the proposed NPMLE is shown through application to synthetic datasets and real datasets.
