基于核方法的实验研究外部有效性
Kernel-based External Validity of Experimental Studies
AI总结：
该研究提出基于核方法的实验外部有效性框架，建立研究规模与MMD分位数的幂律关系，可估计外部有效研究规模，适用于带并列的排名等问题，经实验验证并发布代码。
摘要：
研究的外部有效性是指其结果推广到未见过的实验条件（如数据集）的程度。我们将研究结果与其研究问题形式化，定义外部有效性为研究的两个独立且规模相同的复制品产生的结果的最大均值差异（MMD）低于给定容差的概率。这种以结果空间选择和适用于MMD的再生核选择为核心的方法，足够灵活，可应用于多种不同类型的结果和研究问题。此外，我们建立了研究规模与MMD分位数之间的渐近幂律关系，用于从小规模初步研究估计外部有效研究所需的规模。我们将框架应用于带并列的排名问题，并讨论适用于不同研究问题的核。对现有案例研究和模拟新设计研究的实验表明了该框架的适用性。我们将代码作为genexpy Python模块的一部分发布。
英文摘要：
The external validity of a study is the extent to which its findings generalize to unseen experimental conditions, such as datasets. We formalize the results of a study together with its research question, and define external validity as the probability that two independent and same-sized replications of the study produce results whose maximum mean discrepancy (MMD) is below a given tolerance. This approach, centered on the choice of a space of results and of an appropriate reproducing kernel for the MMD, is flexible enough to be deployed on multiple different types of results and research questions. Additionally, we establish an asymptotic power-law relation between the size of a study and the quantiles of the MMD, which we use to estimate the desired size of an externally valid study from a small-scale preliminary study. We deploy our framework to rankings with ties and discuss kernels suited to different research questions. Experiments on existing case studies and on a simulated newly designed study illustrate the applicability of the framework. We release our code as part of the genexpy Python module.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
