众包中的质量控制:质量属性、评估技术和保证措施的调查
Quality Control in Crowdsourcing: A Survey of Quality Attributes, Assessment Techniques and Assurance Actions
摘要:
众包使人们能够利用潜在的大群体的智力和智慧来解决问题。众包处理的常见问题是标记图像,翻译或转录文本,提供意见或想法,以及类似的-所有计算机不擅长或甚至可能完全失败的任务。然而,将人类引入计算和/或日常工作也在质量控制方面提出了关键的新挑战,因为人群通常由具有未知和非常多样化的能力,技能,兴趣,个人目标和技术资源的人组成。本调查从多个维度研究众包环境下的质量,以定义和描述其特征,并了解当前的技术水平。具体而言,本调查得出了众包任务的质量模型,确定了可用于评估模型属性的方法和技术,以及有助于预防和缓解质量问题的行动和策略。分析这些功能是如何支持的最先进的进一步确定开放的问题,并通知未来的热门研究方向的前景。
英文摘要:
Crowdsourcing enables one to leverage on the intelligence and wisdom of potentially large groups of individuals toward solving problems. Common problems approached with crowdsourcing are labeling images, translating or transcribing text, providing opinions or ideas, and similar - all tasks that computers are not good at or where they may even fail altogether. The introduction of humans into computations and/or everyday work, however, also poses critical, novel challenges in terms of quality control, as the crowd is typically composed of people with unknown and very diverse abilities, skills, interests, personal objectives and technological resources. This survey studies quality in the context of crowdsourcing along several dimensions, so as to define and characterize it and to understand the current state of the art. Specifically, this survey derives a quality model for crowdsourcing tasks, identifies the methods and techniques that can be used to assess the attributes of the model, and the actions and strategies that help prevent and mitigate quality problems. An analysis of how these features are supported by the state of the art further identifies open issues and informs an outlook on hot future research directions.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。