不要怪数据，要怪模型：理解从主观标注中学习时的噪声与偏差
Don't Blame the Data, Blame the Model: Understanding Noise and Bias When Learning from Subjective Annotations
AI总结：
针对主观任务中聚合标签导致高分歧样本学习难的问题，提出用多基准真值（Multi-GT）方法分类，提升了模型在高分歧样本上的置信度，指出学习效果差源于传统聚合模型而非数据。
摘要：
研究人员已提升了对聚合标签危害的认知，尤其是在人类标注者之间天然存在分歧的主观任务中。本研究表明，仅接收聚合标签的模型在高分歧数据样本上表现出低置信度。以往研究将这类样本视为标注错误，而我们认为高分歧文本样本难以学习的原因，是传统聚合模型在从主观任务中提取有用信号方面表现不佳。受近期证明从原始标注中学习有效性的研究启发，我们探究了使用多基准真值（Multi-GT）方法进行分类。实验显示，该方法提升了模型在高分歧样本上的置信度。
英文摘要：
Researchers have raised awareness about the harms of aggregating labels especially in subjective tasks that naturally contain disagreements among human annotators. In this work we show that models that are only provided aggregated labels show low confidence on high-disagreement data instances. While previous studies consider such instances as mislabeled, we argue that the reason the high-disagreement text instances have been hard-to-learn is that the conventional aggregated models underperform in extracting useful signals from subjective tasks. Inspired by recent studies demonstrating the effectiveness of learning from raw annotations, we investigate classifying using Multiple Ground Truth (Multi-GT) approaches. Our experiments show an improvement of confidence for the high-disagreement instances.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
