使用Simpson's Paragraph发现行为数据中的有趣模式
Using Simpson's Paradox to Discover Interesting Patterns in Behavioral Data
摘要:
我们描述了一个数据驱动的发现方法,利用辛普森悖论发现有趣的模式的行为数据。我们的方法系统地分解数据,以确定人口中的子群体,其行为显着偏离其余人口。给定一个感兴趣的结果和协变量集,该方法遵循三个步骤。首先,它通过特定协变量的条件将数据分解为子组,以最小化子组内结果的变化。接下来,它将结果建模为另一个协变量的线性函数,包括子组和汇总数据。最后,它比较趋势,以确定分解,产生不同的行为从聚集的子组。我们通过将其应用于三个真实世界的行为数据集来说明该方法,包括问答网站Stack Exchange和在线学习平台Khan Academy和Duolingo。
英文摘要:
We describe a data-driven discovery method that leverages Simpson's paradox to uncover interesting patterns in behavioral data. Our method systematically disaggregates data to identify subgroups within a population whose behavior deviates significantly from the rest of the population. Given an outcome of interest and a set of covariates, the method follows three steps. First, it disaggregates data into subgroups, by conditioning on a particular covariate, so as minimize the variation of the outcome within the subgroups. Next, it models the outcome as a linear function of another covariate, both in the subgroups and in the aggregate data. Finally, it compares trends to identify disaggregations that produce subgroups with different behaviors from the aggregate. We illustrate the method by applying it to three real-world behavioral datasets, including Q\&A site Stack Exchange and online learning platforms Khan Academy and Duolingo.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。