LLM作为标注者的偏见：政党线索对大型语言模型标注决策的影响
Bias in LLMs as Annotators: The Effect of Party Cues on Labelling Decision by Large Language Models
AI总结：
通过复现实验发现，LLM作为标注者会利用政党线索判断政治声明并反映训练数据偏见，且比人类更易在温和政党声明上表现出显著偏见。
摘要：
人类编码员存在偏见。我们测试了大型语言模型（LLM）作为标注者是否存在类似偏见。通过复现Ennser-Jedenastik和Meyer（2018）的实验，我们发现证据表明LLM会使用政治信息，特别是政党线索，来判断政治声明。LLM不仅利用相关信息，根据政党线索将声明情境化为正面、负面或中立，它们还反映了其训练所基于的人类生成数据的偏见。我们还发现，与人类仅在面临极端政党声明时才表现出偏见不同，即使面对中左翼和中右翼政党的声明提示，LLM也表现出显著偏见。结论部分讨论了这些发现的启示。
英文摘要：
Human coders are biased. We test similar biases in Large Language Models (LLMs) as annotators. By replicating an experiment run by Ennser-Jedenastik and Meyer (2018), we find evidence that LLMs use political information, and specifically party cues, to judge political statements. Not only do LLMs use relevant information to contextualize whether a statement is positive, negative, or neutral based on the party cue, they also reflect the biases of the human-generated data upon which they have been trained. We also find that unlike humans, who are only biased when faced with statements from extreme parties, LLMs exhibit significant bias even when prompted with statements from center-left and center-right parties. The implications of our findings are discussed in the conclusion.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
