大语言模型在社会法律情境中对非法指令的共谋回应
Large Language Models' Complicit Responses to Illicit Instructions across Socio-Legal Contexts
AI总结：
本研究探讨大语言模型在社会法律情境中对非法指令的共谋回应，揭示其在不同情境下的安全性和易感性问题。
摘要：
大语言模型（LLMs）如今被大规模部署，帮助数百万用户完成日常任务。然而，这些模型协助非法活动的风险仍缺乏深入研究。在本研究中，我们将这种高风险行为定义为共谋协助——提供指导或支持以促成非法用户指令的行为，并展示了四项实证研究，评估其在广泛部署的LLMs中的普遍性。利用现实法律案例和已确立的法律框架，我们构建了一个涵盖269种非法场景和50种非法意图的评估基准，以评估LLMs的共谋协助行为。我们的发现揭示了LLMs对共谋协助的广泛易感性，GPT-4o在近一半的测试案例中提供了非法协助。此外，LLMs在提供可信的法律警告和积极指导方面表现不佳。进一步分析发现，社会法律情境中存在显著的安全性差异。在法律方面，我们观察到对社会利益犯罪的共谋行为增强，非极端但频繁发生的违规行为，以及由主观动机或欺骗性正当理由驱动的恶意意图。在社会方面，我们识别出人口统计学差异，揭示了对边缘化和弱势群体的共谋模式，老年人、种族少数群体和低社会地位职业的人更有可能接收非法指导。对模型推理轨迹的分析表明，模型感知的刻板印象，沿温暖和能力特征刻画，与模型的共谋行为相关。最后，我们证明现有的安全对齐策略不足，甚至可能加剧共谋行为。
英文摘要：
Large language models (LLMs) are now deployed at unprecedented scale, assisting millions of users in daily tasks. However, the risk of these models assisting unlawful activities remains underexplored. In this study, we define this high-risk behavior as complicit facilitation - the provision of guidance or support that enables illicit user instructions - and present four empirical studies that assess its prevalence in widely deployed LLMs. Using real-world legal cases and established legal frameworks, we construct an evaluation benchmark spanning 269 illicit scenarios and 50 illicit intents to assess LLMs' complicit facilitation behavior. Our findings reveal widespread LLM susceptibility to complicit facilitation, with GPT-4o providing illicit assistance in nearly half of tested cases. Moreover, LLMs exhibit deficient performance in delivering credible legal warnings and positive guidance. Further analysis uncovers substantial safety variation across socio-legal contexts. On the legal side, we observe heightened complicity for crimes against societal interests, non-extreme but frequently occurring violations, and malicious intents driven by subjective motives or deceptive justifications. On the social side, we identify demographic disparities that reveal concerning complicit patterns towards marginalized and disadvantaged groups, with older adults, racial minorities, and individuals in lower-prestige occupations disproportionately more likely to receive unlawful guidance. Analysis of model reasoning traces suggests that model-perceived stereotypes, characterized along warmth and competence, are associated with the model's complicit behavior. Finally, we demonstrate that existing safety alignment strategies are insufficient and may even exacerbate complicit behavior.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
