arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2408.08212cs.CLcs.CY
收藏

隐性偏见:语言模型中社会观点对隐性与显性意见的未对齐严重程度

Covert Bias: The Severity of Social Views' Unalignment in Language Models Towards Implicit and Explicit Opinion

King Saud University(沙特国王大学);College of Computer and Information Sciences(计算机与信息科学学院)

Abeer Aldayel, Areej Alokaili, Rehab Alahmadi

AI总结:

该研究聚焦隐性语言对大语言模型偏见放大的影响,通过下游任务与压力测试发现模型对显性对立意见存在偏见,且未对齐模型回复欠谨慎,提出需加入不确定性标记提升模型在社会微妙话题上的可靠性。

摘要:

尽管近期已有多种方法被用于偏见识别,但人们对未明确传达观点的隐性语言如何影响大语言模型中的偏见放大仍知之甚少。为检验针对某一观点的偏见严重程度,我们评估了两项下游任务的表现,任务中使用了社会群体的隐性与显性知识。首先,我们通过在过度偏见场景的极端案例中使用有偏见模型,提出了一种压力测试评估方法。随后,我们评估了当大语言模型(LLMs)面对与冲突观点对齐的隐性和显性意见时,如何在语言层面进行校准。我们的发现揭示了LLM在识别隐性与显性意见时的表现差异,普遍存在对对立立场的显性意见存在偏见的倾向。此外,与未对齐的(零样本)基础模型相比,偏见对齐模型会使用不确定性表述生成更谨慎的回复。未对齐模型直接、欠谨慎的回复表明,需要通过加入不确定性标记进一步优化决策果断性,以提升其可靠性,尤其是在主观性强、社会层面微妙的话题上。

英文摘要:

While various approaches have recently been studied for bias identification, little is known about how implicit language that does not explicitly convey a viewpoint affects bias amplification in large language models. To examine the severity of bias toward a view, we evaluated the performance of two downstream tasks where the implicit and explicit knowledge of social groups were used. First, we present a stress test evaluation by using a biased model in edge cases of excessive bias scenarios. Then, we evaluate how LLMs calibrate linguistically in response to both implicit and explicit opinions when they are aligned with conflicting viewpoints. Our findings reveal a discrepancy in LLM performance in identifying implicit and explicit opinions, with a general tendency of bias toward explicit opinions of opposing stances. Moreover, the bias-aligned models generate more cautious responses using uncertainty phrases compared to the unaligned (zero-shot) base models. The direct, incautious responses of the unaligned models suggest a need for further refinement of decisiveness by incorporating uncertainty markers to enhance their reliability, especially on socially nuanced topics with high subjectivity.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑