PolBiX：通过X-委婉语检测大语言模型在事实核查中的政治偏见
PolBiX: Detecting LLMs' Political Bias in Fact-Checking through X-phemisms
AI总结：
本文通过构造德语委婉语/贬义语最小对立陈述对，系统评估六个大语言模型在事实核查中的政治偏见，发现评判性词语比政治倾向更能影响真实性判断，且提示中呼吁客观主义无法缓解偏见。
摘要：
大语言模型越来越多地被用于需要客观评估的应用中，而这种客观性可能受到政治偏见的损害。许多研究发现大语言模型偏好左倾立场，但对事实核查等下游任务的影响仍未得到充分探索。在本研究中，我们通过在德语陈述中替换委婉语或贬义语来系统性地考察政治偏见。我们构建了事实等价但在政治内涵上不同的最小对立陈述对，以评估大语言模型在将其分类为真或假时的一致性。我们评估了六个大语言模型，发现与政治倾向相比，评判性词语的存在对真实性评估的影响更为显著。虽然少数模型表现出政治偏见倾向，但在提示中明确呼吁客观主义并不能缓解这种偏见。警告：本文包含可能令人反感或不安的内容。
英文摘要：
Large Language Models are increasingly used in applications requiring objective assessment, which could be compromised by political bias. Many studies found preferences for left-leaning positions in LLMs, but downstream effects on tasks like fact-checking remain underexplored. In this study, we systematically investigate political bias through exchanging words with euphemisms or dysphemisms in German claims. We construct minimal pairs of factually equivalent claims that differ in political connotation, to assess the consistency of LLMs in classifying them as true or false. We evaluate six LLMs and find that, more than political leaning, the presence of judgmental words significantly influences truthfulness assessment. While a few models show tendencies of political bias, this is not mitigated by explicitly calling for objectivism in prompts. Warning: This paper contains content that may be offensive or upsetting.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
