CrowdHub：扩展众包平台，用于任务设计的受控评估
CrowdHub: Extending crowdsourcing platforms for the controlled evaluation of tasks designs
摘要：
我们提出了CrowdHub，一个工具，用于在众包平台上运行任务设计的系统评估。我们的目标是支持评估过程，避免潜在的实验偏差，根据我们的实证研究，在不受控制的环境中，这种偏差可能导致收集的数据集的效用损失38%。使用CrowdHub，研究人员可以映射他们的实验设计，并自动化管理任务执行的复杂过程，同时控制返回的工人和人群人口统计，从而减少偏见，提高收集数据的实用性，并更有效地利用有限的受试者。
英文摘要：
We present CrowdHub, a tool for running systematic evaluations of task designs on top of crowdsourcing platforms. The goal is to support the evaluation process, avoiding potential experimental biases that, according to our empirical studies, can amount to 38% loss in the utility of the collected dataset in uncontrolled settings. Using CrowdHub, researchers can map their experimental design and automate the complex process of managing task execution over time while controlling for returning workers and crowd demographics, thus reducing bias, increasing utility of collected data, and making more efficient use of a limited pool of subjects.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
