arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2602.21068stat.MEmath.STstat.APstat.TH
收藏

通过顺序假设检验检测效应发生的位置

Detecting Where Effects Occur by Testing Hypotheses in Order

MIT GOV/LAB(麻省理工学院政府实验室)

Jake Bowers, David Kim, Nuole Chen

AI总结:

提出一种自上而下的树形假设检验方法,利用实验的行政层级结构控制族系错误率,并通过误差负荷诊断判断是否需要调整,在教育试验和就业培训研究中验证了其有效性。

摘要:

公共政策的实验评估通常会在多个站点或区块内随机化干预。一旦报告了总体效应,对行动而言重要的问题是效应发生在哪里。标准的多次检验校正由于忽略了实验的组织方式(区块嵌套于队列、站点和区域)而功效较低。我们将假设组织成遵循这种行政结构的树,并自上而下地进行检验,仅当父节点原假设被拒绝时才进入子分支。我们证明,停止规则和有效的节点级检验足以弱控制族系错误率(FWER)。同一过程是否也能强控制FWER取决于一个在数据观测前即可计算的单一量:误差负荷,它总结了拒绝概率沿树中路径的累积方式。这个诊断指标预先告诉分析者,仅从设计量出发,未调整的过程是否控制FWER或需要调整。在25个区块随机化的MDRC教育试验中,该指标表明每个试验都不需要调整,因此仅这两个条件就能控制FWER,同时每个检验在完整的名义水平下运行;自上而下的过程检测到了Hommel校正遗漏的单个区块,并定位了自下而上检验无法评估的高层区块组。对于高误差负荷的设计,我们推导了一个自适应α调度,证明它在规则、不规则和修剪的树上控制FWER,并通过模拟确认。相同的诊断指标在需要时发出警告:在一个针对国家职业训练团研究校准的设计中,这是一个约一百个中心的大规模多站点试验,未调整的过程使FWER膨胀,自适应调度恢复控制,而自上而下的检验仍然比自下而上或分层校正检测到更多受影响的站点。

英文摘要:

Experimental evaluations of public policies often randomize a new intervention within many sites or blocks. After an overall statistically significant result is reported, the natural question from a policy maker is: \emph{where} did effects occur? Standard adjustments for multiple testing answer this question with little power because they ignore how the experiment is organized: blocks nest within cohorts, sites, and districts. We organize the hypotheses in the shape of a tree that follows this administrative structure and test them top-down, stopping at any branch where the null is not rejected. A stopping rule and valid tests at each node suffice for weak control of the family-wise error rate (FWER). Whether the unadjusted procedure also controls the FWER in the strong sense depends on an \emph{error load} computable from design quantities before any data are tested; when the load exceeds one, an adaptive $α$-schedule, which we prove controls the FWER on regular and irregular trees without pruning, restores control. [Correction, August 2026: an anonymous referee identified errors in the previous version. The error loads reported in the paper were computed incorrectly; corrected loads exceed 1 in all 25 block-randomized MDRC education trials at the planning effect size $d = 0.20$, so the adjustment the earlier version claimed unnecessary is in fact required. The theorem claiming FWER control under branch pruning and its switching corollary are false as stated and are withdrawn; an exact counterexample attains FWER 0.063 at $α= 0.05$. The detection comparison with the Hommel procedure is under recomputation. A correction notice on page 1 details what changes and what stands; a fully corrected version is in preparation.]

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑