关于异常值和影响效应的决策重要吗？来自358项行为科学元分析的证据
Do decisions about outliers and influential effects matter? Evidence from 358 behavioral science meta-analyses
AI总结：
研究358项行为科学元分析中异常值处理方法的影响，通过四种预先注册的处理方法与两种估计器结合，对比合并效应等三个结果，发现处理方法对均值影响小，但会逆转部分元分析的统计显著性等，为相关人员提供参考。
AI中文摘要：
元分析者经常面临看起来过大或极端的估计值，如何处理这些值取决于评审者的判断。虽然检测此类估计值的方法是已知的，但缺乏对替代处理选择可能改变元分析结论程度的明智评估。我们通过分析358项行为科学元分析中四种预先注册的处理方法的效果来填补这一空白。每种异常值处理方法由两种估计器估计，并与“不做处理”的基线在合并效应、统计显著性以及效应是否达到最小感兴趣效应大小（|d| >= 0.20）这三个结果上进行比较。我们的整个分析和比较流程都预先注册了。替代异常值处理方法对元分析均值影响很小，科恩d的中位数绝对变化最多为0.047，通常更小。然而，这四种处理方法中的至少一种与其中一种估计器相结合，会使11.5%的元分析的统计显著性和15.9%的最小感兴趣效应评估发生逆转。 Winsorizing的影响最小，DFBETAS的影响最大。分类变化几乎完全出现在已经接近决策边界的结果中；强显著结果基本不变。这些发现为应用元分析者、方法专家和评审者提供了一个参考点，说明这种未充分报告的选择有多重要，并为元分析者公开预先指定其方法和处理方法提供了另一个理由。
英文摘要：
Meta-analysts routinely face estimates that look too large or extreme. Yet, how to handle them is left to the reviewer's judgment. The methods for detecting such estimates are well known. What is missing is an informed assessment of how much alternative handling choices might change a meta-analysis' conclusions. We fill this gap by analyzing the effects of four pre-registered handling treatments across 358 behavioral science meta-analyses with at least ten estimates. Each outlier handling treatment is estimated by two estimators (random effects and unrestricted weighted least squares), and compared to the 'do-nothing' baseline on three outcomes: the pooled effect, statistical significance, and whether the effect reaches the smallest effect size of interest (|d| >= 0.20). Our entire analysis and comparison pipelines were pre-registered. Alternative outlier handling treatments have little effect on the meta-analysis mean as the median absolute change in Cohen's d is at most 0.047 and often much less. Yet, at least one of these four treatments in combination with one of these estimators reverses the statistical significance of 11.5% of meta-analyses and the smallest-effect-of-interest assessment in 15.9%. Winsorizing has the least effect and DFBETAS the most. Categorical changes are found almost entirely among results already close to the decision boundary; strongly significant results essentially never change. These findings give applied meta-analysts, methods specialists, and reviewers a reference point for how much this under-reported choice matters and provide yet another reason for meta-analysts to publicly pre-specify their methods and handling treatments.
