Can Crowds serve as useful allies in policy design? How do non-expert Crowds perform relative to experts in the assessment of policy measures? Does the geographic location of non-expert Crowds, with relevance to the policy context, alter the performance of non-experts Crowds in the assessment of policy measures? In this work, we investigate these questions by undertaking experiments designed to replicate expert policy assessments with non-expert Crowds recruited from Virtual Labor Markets. We use a set of ninety-six climate change adaptation policy measures previously evaluated by experts in the Netherlands as our control condition to conduct experiments using two discrete sets of non-expert Crowds recruited from Virtual Labor Markets. We vary the composition of our non-expert Crowds along two conditions: participants recruited from a geographical location directly relevant to the policy context and participants recruited at-large. We discuss our research methods in detail and provide the findings of our experiments.
A Multimodal, Multilingual, and Multidimensional Pipeline for Fine-grained Crowdsourcing Earthquake Damage Evaluation
New York University（纽约大学）；University of South Florida（佛罗里达州立大学）；Google LLC（谷歌公司）；University of California, Santa Barbara（加州大学圣巴巴拉分校）；Massachusetts Institute of Technology（麻省理工学院）
A Policy Report Evaluating the National Assessment Program for Literacy and Numeracy (Naplan) Reform in Australia: The Impacts of High Stakes Assessment on Students
Informing AI Policy Assessment using Large-Scale Simulation of Interventions
Northwestern University USA；University of Amsterdam, The Netherlands \& University of Hohenheim Germany；University of Amsterdam The Netherlands；Northwestern University；University of Amsterdam, The Netherlands \& University of Hohenheim；University of Amsterdam
A Multilayer Perceptron-based Fast Sunlight Assessment for the Conceptual Design of Residential Neighborhoods under Chinese Policy
Glodon Company Limited（广联达科技股份有限公司）；Tsinghua University（清华大学）；Institute for Urban Governance and Sustainable Development, Tsinghua University（清华大学城市治理与可持续发展研究院）
