arXivDaily arXiv每日学术速递 周一至周五更新
​通知:联邦节假日调休:哥伦布日 / 原住民日(Columbus Day / Indigenous Peoples' Day),今日(13日)无更新。
arXiv 2604.11120cs.AI

不受欢迎的人:针对具有人格特征的LLM,单一方法的安全性评估是不完整的

Persona Non Grata: Single-Method Safety Evaluation Is Incomplete for Persona-Imbued LLMs

Carnegie Mellon University(卡内基梅隆大学);Fujitsu Research of America Inc.(富士通美国研究公司)

Wenkai Li, Fan Yang, Shaunak A. Mehta, Koichi Onoue

AI总结:

本文指出,针对具有人格特征的LLM,仅通过提示进行的安全性评估不完整,因为提示和激活引导暴露了不同的、依赖于架构的漏洞特征。研究显示,激活引导的漏洞无法通过提示侧排名预测,且在不同架构模型中表现各异。

摘要:

人格植入会定制LLM的行为,但安全性评估几乎总是单独研究基于提示的人格。我们证明这是不完整的:提示和激活引导暴露了不同的、依赖于架构的漏洞特征,仅测试一种方法可能错过模型的主要失败模式。在五个标准模型的5,568个判断条件下,系统提示下的人格危险排名在所有架构中保持一致(ρ=0.71-0.96),但激活引导的漏洞却急剧偏离且无法从提示侧排名预测:Llama-3.1-8B显著更易受激活引导攻击,而Gemma-3-27B和Qwen3.5则更易受提示攻击。最引人注目的例子是“亲社会人格悖论”:在Llama-3.1-8B上,P12(高责任心+高亲和力)在提示下是最安全的人格之一,却成为最高ASR激活引导人格(ASR ~0.818)。这种倒置在系数消除和匹配强度校准后仍保持稳定,并在DeepSeek-R1-Distill-Qwen-32B上重复。一个特质拒绝对齐框架,在Llama-3.1-8B上,责任心强烈反向对齐于拒绝,提供部分几何解释。推理仅提供部分保护:两个32B推理模型在提示侧达到15-18%的ASR,激活引导在基线易感性和人格特定漏洞上显著分离。启发式跟踪诊断表明,更安全的模型保留了更强的政策回忆和自我纠正行为,而不仅仅是更长的推理。

英文摘要:

Personality imbuing customizes LLM behavior, but safety evaluations almost always study prompt-based personas alone. We show this is incomplete: prompting and activation steering expose *different*, architecture-dependent vulnerability profiles, and testing with only one method can miss a model's dominant failure mode. Across 5,568 judged conditions on four standard models from three architecture families, persona danger rankings under system prompting are preserved across all architectures ($ρ= 0.71$--$0.96$), but activation-steering vulnerability diverges sharply and cannot be predicted from prompt-side rankings: Llama-3.1-8B is substantially more AS-vulnerable, whereas Gemma-3-27B and Qwen3.5 are more vulnerable to prompting. The most striking illustration of this divergence is the *prosocial persona paradox*: on Llama-3.1-8B, P12 (high conscientiousness + high agreeableness) is among the safest personas under prompting yet becomes the highest-ASR activation-steered persona (ASR ~0.818). This is an inversion robust to coefficient ablation and matched-strength calibration, and replicated on DeepSeek-R1-Distill-Qwen-32B. A trait refusal alignment framework, in which conscientiousness is strongly anti-aligned with refusal on Llama-3.1-8B, offers a partial geometric account. Reasoning provides only partial protection: two 32B reasoning models reach 15--18% prompt-side ASR, and activation steering separates them sharply in both baseline susceptibility and persona-specific vulnerability. Heuristic trace diagnostics suggest that the safer model retains stronger policy recall and self-correction behavior, not merely longer reasoning.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑