BadJudge：LLM-as-a-Judge的后门漏洞
BadJudge: Backdoor Vulnerabilities of LLM-as-a-Judge
AI总结：
本文揭示了LLM-as-a-Judge评估机制的后门漏洞，攻击者通过控制候选与评估模型实现分数虚高，并提出利用模型合并作为低成本的防御方案以保持性能并降低攻击成功率。
摘要：
本文提出了一种针对LLM-as-a-Judge评估机制的新型后门威胁，其中攻击者同时控制候选模型和评估器模型。被植入后门的评估器通过不公平地向攻击者分配虚高的分数，使良性用户受害。仅在1%的评估器训练数据中植入简单的单token后门，就能使攻击者的得分相对于其合法得分增加两倍。我们系统地划分了对应三种现实场景的数据访问级别：(1) 网络投毒，(2) 恶意标注者，(3) 权重投毒。这些场景反映了从弱到强的数据访问权限升级，且与攻击严重程度高度相关。在最弱的假设——网络投毒(1)下，攻击者仍能诱导20%的分数虚高。同样，在权重投毒(3)场景中，更强的假设使攻击者能将其分数从1.5/5虚高至4.9/5。该后门威胁可泛化至不同的评估器架构、触发器设计、评估任务和投毒率。通过投毒10%的评估器训练数据，我们控制毒性评判器（Guardrails）在89%的情况下将有毒提示误分类为无毒，并使RAG中的文档重排序评判器在97%的情况下将投毒文档排在首位。LLM-as-a-Judge独特地处于伦理与技术的交汇点，其中误导模型选择和评估的社会影响限制了可用的防御工具。面对这些挑战，模型合并成为一种抵消后门的原则性工具，在保持SOTA性能的同时将ASR降至接近0%。模型合并的低计算成本及其与当前LLM Judge训练流程的便捷整合，使其成为LLM-as-a-Judge场景中缓解后门的有前景的途径。
英文摘要：
This paper proposes a novel backdoor threat attacking the LLM-as-a-Judge evaluation regime, where the adversary controls both the candidate and evaluator model. The backdoored evaluator victimizes benign users by unfairly assigning inflated scores to adversary. A trivial single token backdoor poisoning 1% of the evaluator training data triples the adversary's score with respect to their legitimate score. We systematically categorize levels of data access corresponding to three real-world settings, (1) web poisoning, (2) malicious annotator, and (3) weight poisoning. These regimes reflect a weak to strong escalation of data access that highly correlates with attack severity. Under the weakest assumptions - web poisoning (1), the adversary still induces a 20% score inflation. Likewise, in the (3) weight poisoning regime, the stronger assumptions enable the adversary to inflate their scores from 1.5/5 to 4.9/5. The backdoor threat generalizes across different evaluator architectures, trigger designs, evaluation tasks, and poisoning rates. By poisoning 10% of the evaluator training data, we control toxicity judges (Guardrails) to misclassify toxic prompts as non-toxic 89% of the time, and document reranker judges in RAG to rank the poisoned document first 97% of the time. LLM-as-a-Judge is uniquely positioned at the intersection of ethics and technology, where social implications of mislead model selection and evaluation constrain the available defensive tools. Amidst these challenges, model merging emerges as a principled tool to offset the backdoor, reducing ASR to near 0% whilst maintaining SOTA performance. Model merging's low computational cost and convenient integration into the current LLM Judge training pipeline position it as a promising avenue for backdoor mitigation in the LLM-as-a-Judge setting.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
