arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2503.09347cs.CLcs.AI
收藏

更安全还是更幸运?LLM作为安全评估者对伪影不具鲁棒性

Safer or Luckier? LLMs as Safety Evaluators Are Not Robust to Artifacts

Cohere

Hongyu Chen, Seraphina Goldfarb-Tarrant

AI总结:

本研究评估11个LLM评判模型在安全评估中的鲁棒性,发现输入伪影(如道歉措辞)可致偏好偏差达98%,且大模型不更鲁棒;陪审团聚合虽改善鲁棒性和人类一致性,但伪影敏感性仍存。

摘要:

大型语言模型(LLM)越来越多地被用作自动化评估器来评估生成内容的安全性,但它们在扮演这一角色时的可靠性仍不确定。本研究在关键安全领域评估了11个不同的LLM评判模型,考察了三个关键方面:重复评判任务中的自我一致性、与人类判断的一致性,以及对诸如道歉性或冗长措辞等输入伪影的敏感性。我们的研究结果显示,LLM评判者中的偏见可能显著扭曲关于哪个内容来源更安全的最终裁决,从而削弱比较评估的有效性。值得注意的是,仅道歉性语言伪影就能使评估者的偏好偏差高达98%。与预期相反,较大的模型并不始终表现出更强的鲁棒性,而较小的模型有时对特定伪影表现出更高的抵抗力。为缓解LLM评估器的鲁棒性问题,我们研究了基于陪审团的评估方法,即聚合多个模型的决策。尽管这种方法既提高了鲁棒性,也增强了与人类判断的一致性,但即使在最佳的陪审团配置下,伪影敏感性仍然存在。这些结果凸显了迫切需要多样化、抗伪影的方法来确保可靠的安全评估。

英文摘要:

Large Language Models (LLMs) are increasingly employed as automated evaluators to assess the safety of generated content, yet their reliability in this role remains uncertain. This study evaluates a diverse set of 11 LLM judge models across critical safety domains, examining three key aspects: self-consistency in repeated judging tasks, alignment with human judgments, and susceptibility to input artifacts such as apologetic or verbose phrasing. Our findings reveal that biases in LLM judges can significantly distort the final verdict on which content source is safer, undermining the validity of comparative evaluations. Notably, apologetic language artifacts alone can skew evaluator preferences by up to 98\%. Contrary to expectations, larger models do not consistently exhibit greater robustness, while smaller models sometimes show higher resistance to specific artifacts. To mitigate LLM evaluator robustness issues, we investigate jury-based evaluations aggregating decisions from multiple models. Although this approach both improves robustness and enhances alignment to human judgements, artifact sensitivity persists even with the best jury configurations. These results highlight the urgent need for diversified, artifact-resistant methodologies to ensure reliable safety assessments.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑