arXiv 2402.153022024-11-16cs.CLcs.CR

LLM 以指令为中心的响应有多（不）道德？揭示安全护栏对有害查询的脆弱性

How (un)ethical are instruction-centric responses of LLMs? Unveiling the vulnerabilities of safety guardrails to harmful queries

Indian Institute of Technology Kharagpur（印度理工学院卡拉格普尔分校）；Singapore University of Technology and Design（新加坡科技设计大学）

Somnath Banerjee, Sayan Layek, Rima Hazra, Animesh Mukherjee

备注：: Accepted at AAAI Conference on Web and Social Media (ICWSM) 2025. [Dataset](https://huggingface.co/datasets/SoftMINER-Group/TechHazardQA)

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

本研究探讨了大型语言模型在生成指令为中心响应（如伪代码）时易产生不道德内容的安全漏洞，引入TechHazardQA数据集评估多个模型，发现该方式显著增加不道德响应生成率，且ROME模型编辑技术会进一步加剧此风险。

摘要：

在本研究中，我们探讨了围绕大型语言模型（LLM）安全与伦理使用的一个日益增长的担忧。尽管这些模型具有潜力，但可以通过各种复杂的方法（包括“越狱”技术和定向操纵）诱骗它们生成有害或不道德的内容。我们的工作聚焦于一个具体问题：通过要求 LLM 生成以指令为中心的响应（如伪代码、程序或软件代码片段），而不是普通文本，它们能在多大程度上被误导。为了调查这个问题，我们引入了 TechHazardQA，这是一个包含复杂查询的数据集，这些查询应以文本和以指令为中心的格式（例如伪代码）进行回答，旨在识别不道德响应的触发因素。我们查询了一系列 LLM——Llama-2-13b、Llama-2-7b、Mistral-V2 和 Mistral 8X7B——并要求它们生成文本和以指令为中心的响应。在评估方面，我们报告了有害性评分指标以及 GPT-4 和人类的判断。总体而言，我们观察到，要求 LLM 生成以指令为中心的响应会使各模型的不道德响应生成率提高约 2-38%。作为额外目标，我们研究了使用 ROME 技术进行模型编辑的影响，这进一步增加了生成不良内容的倾向。特别是，要求经过编辑的 LLM 生成以指令为中心的响应，会使不同模型的不道德响应生成率进一步增加约 3-16%。

英文摘要：

In this study, we tackle a growing concern around the safety and ethical use of large language models (LLMs). Despite their potential, these models can be tricked into producing harmful or unethical content through various sophisticated methods, including 'jailbreaking' techniques and targeted manipulation. Our work zeroes in on a specific issue: to what extent LLMs can be led astray by asking them to generate responses that are instruction-centric such as a pseudocode, a program or a software snippet as opposed to vanilla text. To investigate this question, we introduce TechHazardQA, a dataset containing complex queries which should be answered in both text and instruction-centric formats (e.g., pseudocodes), aimed at identifying triggers for unethical responses. We query a series of LLMs -- Llama-2-13b, Llama-2-7b, Mistral-V2 and Mistral 8X7B -- and ask them to generate both text and instruction-centric responses. For evaluation we report the harmfulness score metric as well as judgements from GPT-4 and humans. Overall, we observe that asking LLMs to produce instruction-centric responses enhances the unethical response generation by ~2-38% across the models. As an additional objective, we investigate the impact of model editing using the ROME technique, which further increases the propensity for generating undesirable content. In particular, asking edited LLMs to generate instruction-centric responses further increases the unethical response generation by ~3-16% across the different models.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。