阴影自我:大型语言模型代理中的内在价值偏差
The Shadow Self: Intrinsic Value Misalignment in Large Language Model Agents
AI总结:
研究提出IMPRESS框架,系统评估大型语言模型代理中的内在价值偏差风险,发现其在不同模型中普遍存在,且受上下文化和框架机制显著影响。
摘要:
大型语言模型(LLM)代理在扩展自主性后解锁了新的能力,但也引入了更高的LLM安全挑战。特别是,一个LLM代理可能会追求偏离人类价值观和伦理规范的目标,这种风险被称为价值偏差。现有的评估主要集中在对显性有害输入的响应或对系统故障的鲁棒性上,而现实、完全无害且代理化设置中的价值偏差仍鲜有研究。为填补这一空白,我们首先正式化了失控风险并识别出此前未充分探讨的内在价值偏差(Intrinsic VM)。然后,我们引入IMPRESS(在现实场景集中的内在价值偏差探测框架),这是一个用于系统评估此风险的场景驱动框架。在我们的框架下,我们构建了由现实、完全无害且上下文化的场景组成的基准,使用多阶段的LLM生成流水线并严格的质量控制。我们在21种最先进的LLM代理上评估了内在VM,并发现它在模型中普遍存在且广泛观察到的安全风险。此外,偏差率因动机、风险类型、模型规模和架构而异。虽然解码策略和超参数只表现出微小影响,但上下文化和框架机制显著塑造了偏差行为。最后,我们进行人类验证以验证我们的自动判断并评估现有缓解策略,如安全提示和护栏,这些策略显示出不稳定或有限的有效性。我们进一步展示了IMPRESS在AI生态系统中的关键使用案例。我们的代码和基准将在接受后公开发布。
英文摘要:
Large language model (LLM) agents with extended autonomy unlock new capabilities, but also introduce heightened challenges for LLM safety. In particular, an LLM agent may pursue objectives that deviate from human values and ethical norms, a risk known as value misalignment. Existing evaluations primarily focus on responses to explicit harmful input or robustness against system failure, while value misalignment in realistic, fully benign, and agentic settings remains largely underexplored. To fill this gap, we first formalize the Loss-of-Control risk and identify the previously underexamined Intrinsic Value Misalignment (Intrinsic VM). We then introduce IMPRESS (Intrinsic Value Misalignment Probes in REalistic Scenario Set), a scenario-driven framework for systematically assessing this risk. Following our framework, we construct benchmarks composed of realistic, fully benign, and contextualized scenarios, using a multi-stage LLM generation pipeline with rigorous quality control. We evaluate Intrinsic VM on 21 state-of-the-art LLM agents and find that it is a common and broadly observed safety risk across models. Moreover, the misalignment rates vary by motives, risk types, model scales, and architectures. While decoding strategies and hyperparameters exhibit only marginal influence, contextualization and framing mechanisms significantly shape misalignment behaviors. Finally, we conduct human verification to validate our automated judgments and assess existing mitigation strategies, such as safety prompting and guardrails, which show instability or limited effectiveness. We further demonstrate key use cases of IMPRESS across the AI Ecosystem. Our code and benchmark will be publicly released upon acceptance.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。