支持算法需求的标注任务设计：促进工人多样性并减少AI偏见
A Labeling Task Design for Supporting Algorithmic Needs: Facilitating Worker Diversity and Reducing AI Bias
AI总结：
本研究开发了一种支持不同背景工人公平参与的标注任务设计，通过分析75名标注员的数据发现，社区支持和机器反馈能促进工人投入工作并减少ML偏见，进而提出连接标注员、机器与社区的扩展human-in-the-loop方法。
摘要：
关于监督机器学习（ML）的研究建议让不同背景的工人参与训练数据集的标注，以减少算法偏见。此外，为了提升ML性能，需要对图像中的物体进行分类的复杂任务，这进一步使微任务复杂化。本研究旨在开发一种任务设计，不论人们的具体背景或任务难度如何，都能实现公平参与。通过与75名来自不同背景的标注员合作3个月，我们分析了工人的日志数据和相关叙述，以识别任务的障碍和助力。研究结果表明，工人的决策倾向因其背景而异。我们发现，积极帮助工人的社区以及工人感知到的机器反馈能够使人们更容易投入工作。因此，ML的偏见有望得到预期缓解。基于这些发现，我们提出了一种扩展的human-in-the-loop（人在回路）方法，该方法连接标注员、机器和社区，而不是孤立个体工人。
英文摘要：
Studies on supervised machine learning (ML) recommend involving workers from various backgrounds in training dataset labeling to reduce algorithmic bias. Moreover, sophisticated tasks for categorizing objects in images are necessary to improve ML performance, further complicating micro-tasks. This study aims to develop a task design incorporating the fair participation of people, regardless of their specific backgrounds or task's difficulty. By collaborating with 75 labelers from diverse backgrounds for 3 months, we analyzed workers' log-data and relevant narratives to identify the task's hurdles and helpers. The findings revealed that workers' decision-making tendencies varied depending on their backgrounds. We found that the community that positively helps workers and the machine's feedback perceived by workers could make people easily engaged in works. Hence, ML's bias could be expectedly mitigated. Based on these findings, we suggest an extended human-in-the-loop approach that connects labelers, machines, and communities rather than isolating individual workers.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
